Redis 生产环境中的高可用性：Sentinel、集群和 Kubernetes Operator

Redis 是现代应用基础设施的支柱。它充当全球数百万应用程序的缓存、会话存储、消息代理、速率限制器、排行榜引擎和实时分析管道。单个 Redis 实例每秒可以处理数十万次操作，延迟时间亚毫秒，但单个实例也是单点故障。当 Redis 出现故障时，应用程序会遇到级联故障：缓存踩踏淹没后端数据库、会话丢失、速率限制器停止工作、实时功能变暗。对于生产系统来说，构建高度可用的 Redis 部署不是可选的 - 这是一项工程要求。

本指南是对 Redis 高可用性的全面、以生产为中心的深入探讨。我们将介绍 Redis 复制基础知识（异步复制、WAIT 命令和部分重新同步）、用于自动故障转移和服务发现的 Redis Sentinel、用于通过哈希槽分布进行水平扩展的 Redis Cluster、Kubernetes 运算符（Spotahome、OpsTree 和 Redis Enterprise）、持久性策略（RDB 快照、AOF 和混合持久性）、AWS ElastiCache 上的云托管部署、适用于 Redis 的 Azure 缓存和 GCP Memorystore、使用 Rancher 和 Longhorn 的裸机 k3s 部署、内存管理和逐出策略、TLS 加密和基于 ACL 的访问控制、HA 配置中的 Pub/Sub 和 Streams 行为、HA 中的 Redis 模块（RedisJSON、RediSearch、RedisTimeSeries）、用于故障转移弹性的连接池和客户端配置、备份和恢复策略、使用 Redis 进行监控INFO、Prometheus 导出器和 Grafana 仪表板、作为 Redis 兼容替代方案的 Dragonfly 和 KeyDB、通过管道进行性能调整、Lua 脚本和内存优化、常见故障场景和故障排除过程以及容量规划和扩展策略。

Redis 复制基础知识

Redis 复制是构建所有高可用性架构的基础。 Redis 主实例接受写入并将其异步传播到一个或多个副本实例。副本维护主数据集的近实时副本并提供读取查询服务，从而提供数据冗余和读取可扩展性。

与 PostgreSQL 基于 WAL 的流式复制不同，Redis 使用基于命令的复制协议。在主服务器上执行的每个写入命令都被序列化到复制流中并发送到连接的副本，副本针对其本地数据集执行相同的命令。这种方法简单高效，但对一致性有重要影响——由于默认情况下复制是异步的，因此总是存在一个窗口，其中主服务器上已确认的写入尚未到达副本。

异步复制和 WAIT 命令

默认情况下，Redis 复制是完全异步的。主服务器在本地应用后立即确认对客户端的写入，而无需等待任何副本确认接收。这提供了最大的写入吞吐量，但引入了潜在的数据丢失窗口 - 如果主服务器在写入到达任何副本之前崩溃，则该写入就会丢失。

WAIT命令提供同步复制原语。发出写入后，客户端可以调用WAIT numreplicas timeout进行阻塞，直到指定数量的副本已确认写入或超时到期。这并不使 Redis 完全同步 -WAIT只保证副本已收到数据，而不保证数据已持久保存到副本上的磁盘上。然而，它显着减少了数据丢失窗口。

# Write a critical value and wait for 2 replicas to acknowledge
SET order:12345 '{"status":"confirmed","amount":599.99}'
WAIT 2 5000
# Returns the number of replicas that acknowledged within 5000ms
# Returns 0 if no replica acknowledged (timeout or no replicas connected)

有选择地使用WAIT进行关键写入（金融交易、订单确认），同时允许非关键写入（缓存更新、会话刷新）异步进行。每个命令的灵活性避免了完全同步复制的延迟损失。

部分重新同步 (PSYNC)

当副本短暂断开连接（网络故障、重新启动）时，它不需要完整的数据集传输即可重新加入。 Redis 在主服务器上维护复制积压（最近写入命令的循环缓冲区）。当副本重新连接时，它将其复制偏移量发送到主服务器。如果偏移量仍在积压范围内，则主设备仅发送丢失的命令（部分重新同步）。如果偏移量超出积压范围，则会触发完全重新同步，其中涉及生成和传输 RDB 快照。

# redis.conf — Replication backlog configuration
repl-backlog-size 256mb          # Size of the replication backlog buffer
repl-backlog-ttl 3600            # Seconds to retain backlog after last replica disconnects
repl-diskless-sync yes           # Transfer RDB via socket instead of disk (faster for full sync)
repl-diskless-sync-delay 5       # Wait 5s for more replicas before starting diskless sync
repl-diskless-sync-period 0      # No periodic full sync
repl-diskless-load on-empty-db   # Replica loads RDB from socket directly into memory

正确调整复制待办事项的大小至关重要。它应该足够大以容纳在最长的预期副本断开连接期间生成的所有写入命令。对于处理 50MB/s 写入流量的 Redis 实例，256MB 积压事务涵盖大约 5 秒的写入 — 如果您的副本可能离线较长时间，请增加该值。

配置主副本复制

# redis.conf — Master configuration
bind 0.0.0.0
port 6379
protected-mode no
requirepass strong_master_password
masterauth strong_master_password

# Persistence
save 900 1
save 300 10
save 60 10000
appendonly yes
appendfsync everysec
aof-use-rdb-preamble yes

# Replication
repl-backlog-size 256mb
repl-backlog-ttl 3600
repl-diskless-sync yes
min-replicas-to-write 1          # Refuse writes if fewer than 1 replica connected
min-replicas-max-lag 10          # Replica considered disconnected if lag > 10 seconds

# redis.conf — Replica configuration
bind 0.0.0.0
port 6379
protected-mode no
requirepass strong_master_password
masterauth strong_master_password

replicaof master-host 6379
replica-read-only yes
replica-serve-stale-data yes     # Serve (possibly stale) data during sync
replica-priority 100             # Lower values get promoted first by Sentinel

当主设备无法保证副本上的数据持久性时，min-replicas-to-write和min-replicas-max-lag设置会阻止主设备接受写入。这是一个关键的安全网——没有它，网络分区的主服务器会继续接受写入，而当 Sentinel 提升副本时，这些写入将会丢失。

Redis Sentinel：自动故障转移和服务发现

Redis Sentinel 是一个分布式系统，可监控 Redis 主实例和副本实例、检测主实例故障、通过将副本提升为主实例来执行自动故障转移，并提供服务发现，以便客户端始终可以找到当前的主实例。 Sentinel 作为一个单独的进程与 Redis 一起运行，并通过共识协议进行操作 - Sentinel 实例的法定数量必须同意在启动故障转移之前无法访问主服务器。

哨兵配置

Sentinel 至少需要三个实例才能容忍一个 Sentinel 故障并仍维持法定人数。每个 Sentinel 监控同一个 Redis master，并通过八卦协议与其他 Sentinel 进行通信，以就 master 的健康状态达成一致。

# /etc/redis/sentinel.conf — Sentinel instance configuration
port 26379
bind 0.0.0.0
protected-mode no

# Monitor the master named "mymaster" at 10.0.1.10:6379
# The quorum value (2) means 2 Sentinels must agree the master is down
sentinel monitor mymaster 10.0.1.10 6379 2

# Authentication
sentinel auth-pass mymaster strong_master_password

# Timing parameters
sentinel down-after-milliseconds mymaster 5000    # SDOWN after 5s of no PING response
sentinel failover-timeout mymaster 60000          # Max 60s for failover procedure
sentinel parallel-syncs mymaster 1                # Only 1 replica syncs from new master at a time

# Deny script execution for security
sentinel deny-scripts-reconfig yes

# Notification script (called on failover events)
# sentinel notification-script mymaster /opt/redis/notify.sh

# Client reconfiguration script (called when master changes)
# sentinel client-reconfig-script mymaster /opt/redis/reconfig.sh

# Logging
logfile /var/log/redis/sentinel.log
logevel notice

# Enable TLS for Sentinel communication
# tls-port 26379
# port 0
# tls-cert-file /etc/redis/tls/sentinel.crt
# tls-key-file /etc/redis/tls/sentinel.key
# tls-ca-cert-file /etc/redis/tls/ca.crt
# tls-replication yes
# tls-auth-clients optional

Sentinel 的故障检测分两个阶段进行。首先，当单个 Sentinel 在down-after-milliseconds内未收到对 PING 的有效回复时，会将主设备标记为主观关闭 (SDOWN)。然后，当法定数量的 Sentinels 同意主服务器无法访问时，它会被标记为Objectively Down (ODOWN)，并且故障转移过程开始。一个 Sentinel 被选举为故障转移领导者，它选择最佳副本（基于优先级、复制偏移量和 runid），将其提升为主副本，重新配置剩余副本以跟随新主副本，并更新 Sentinel 状态。

Sentinel 服务发现和客户端配置

Sentinel 相对于静态主副本设置的主要优势是服务发现。客户端不会连接到固定的 Redis 地址 - 他们向 Sentinel 询问当前的主地址并订阅故障转移通知。每个主要的 Redis 客户端库都原生支持 Sentinel。

# Node.js — ioredis with Sentinel support
const Redis = require('ioredis');

const redis = new Redis({
  sentinels: [
    { host: '10.0.1.20', port: 26379 },
    { host: '10.0.1.21', port: 26379 },
    { host: '10.0.1.22', port: 26379 }
  ],
  name: 'mymaster',
  password: 'strong_master_password',
  sentinelPassword: 'sentinel_password',
  db: 0,
  retryStrategy(times) {
    const delay = Math.min(times * 200, 5000);
    return delay;
  },
  reconnectOnError(err) {
    const targetError = 'READONLY';
    if (err.message.includes(targetError)) {
      return true; // Reconnect on READONLY error (failover happened)
    }
    return false;
  },
  maxRetriesPerRequest: 3,
  enableReadyCheck: true,
  connectTimeout: 10000,
  lazyConnect: false
});

redis.on('connect', () => console.log('Connected to Redis master'));
redis.on('error', (err) => console.error('Redis error:', err));
redis.on('+switch-master', (msg) => {
  console.log('Master switched:', msg);
});

# Python — redis-py with Sentinel support
from redis.sentinel import Sentinel
import redis

sentinel = Sentinel(
    [('10.0.1.20', 26379), ('10.0.1.21', 26379), ('10.0.1.22', 26379)],
    socket_timeout=5,
    password='strong_master_password',
    sentinel_kwargs={'password': 'sentinel_password'}
)

# Get a connection to the current master (for writes)
master = sentinel.master_for(
    'mymaster',
    socket_timeout=5,
    retry_on_timeout=True,
    db=0
)

# Get a connection to a replica (for reads)
replica = sentinel.slave_for(
    'mymaster',
    socket_timeout=5,
    db=0
)

# Usage
master.set('session:user123', '{"logged_in": true}')
result = replica.get('session:user123')
print(result)

// Go — go-redis with Sentinel support
package main

import (
    "context"
    "fmt"
    "time"

    "github.com/redis/go-redis/v9"
)

func main() {
    ctx := context.Background()

    rdb := redis.NewFailoverClient(&redis.FailoverOptions{
        MasterName:       "mymaster",
        SentinelAddrs:    []string{"10.0.1.20:26379", "10.0.1.21:26379", "10.0.1.22:26379"},
        Password:         "strong_master_password",
        SentinelPassword: "sentinel_password",
        DB:               0,
        DialTimeout:      10 * time.Second,
        ReadTimeout:      5 * time.Second,
        WriteTimeout:     5 * time.Second,
        PoolSize:         50,
        MinIdleConns:     10,
        MaxRetries:       3,
        MinRetryBackoff:  200 * time.Millisecond,
        MaxRetryBackoff:  5 * time.Second,
    })
    defer rdb.Close()

    err := rdb.Set(ctx, "key", "value", 5*time.Minute).Err()
    if err != nil {
        fmt.Printf("Error: %v
", err)
        return
    }

    val, err := rdb.Get(ctx, "key").Result()
    if err != nil {
        fmt.Printf("Error: %v
", err)
        return
    }
    fmt.Printf("key = %s
", val)
}

Redis 集群：使用哈希槽进行水平扩展

Sentinel 为单个数据集提供高可用性，而 Redis Cluster 提供 HA 和水平扩展。 Redis Cluster 使用哈希槽机制将数据跨多个主节点进行分区 - 键空间分为 16,384 个哈希槽，每个主节点负责这些槽的子集。每个主服务器都有一个或多个副本用于故障转移。 该集群共同提供自动分片、内置故障转移以及通过添加节点线性扩展存储和吞吐量的能力。

设置 Redis 集群

# Create a 6-node Redis Cluster (3 masters + 3 replicas)
# Each node needs a redis.conf with cluster-enabled

# redis.conf for each cluster node (adjust port per node)
port 7000
cluster-enabled yes
cluster-config-file nodes-7000.conf
cluster-node-timeout 5000
appendonly yes
appendfsync everysec
aof-use-rdb-preamble yes
requirepass cluster_password
masterauth cluster_password
bind 0.0.0.0
protected-mode no
repl-backlog-size 256mb

# Start all 6 Redis instances
redis-server /etc/redis/7000.conf
redis-server /etc/redis/7001.conf
# ... repeat for all 6 nodes

# Create the cluster
redis-cli --cluster create \
  10.0.1.10:7000 10.0.1.11:7000 10.0.1.12:7000 \
  10.0.1.13:7000 10.0.1.14:7000 10.0.1.15:7000 \
  --cluster-replicas 1 \
  -a cluster_password

# Verify cluster status
redis-cli -c -h 10.0.1.10 -p 7000 -a cluster_password cluster info
redis-cli -c -h 10.0.1.10 -p 7000 -a cluster_password cluster nodes

重新分片和多密钥操作

重新分片在主节点之间移动哈希槽，以在添加或删除节点后重新平衡数据。在重新分片期间，迁移槽中的键可能会收到 ASK 重定向，客户端会透明地处理该重定向。

# Add a new node to the cluster
redis-cli --cluster add-node 10.0.1.16:7000 10.0.1.10:7000 -a cluster_password

# Reshard slots to the new node
redis-cli --cluster reshard 10.0.1.10:7000 \
  --cluster-from all \
  --cluster-to NEW_NODE_ID \
  --cluster-slots 4096 \
  --cluster-yes \
  -a cluster_password

# Rebalance the cluster automatically
redis-cli --cluster rebalance 10.0.1.10:7000 -a cluster_password

# Check cluster slot distribution
redis-cli -c -h 10.0.1.10 -p 7000 -a cluster_password cluster slots

Redis 集群中的多密钥操作才有效。使用哈希标签确保相关键映射到同一插槽：{user:123}.profile和{user:123}.sessions都在user:123上哈希，保证它们落在同一节点上。这使得 MGET、MSET、事务和 Lua 脚本能够跨相关键。

# Hash tags ensure these keys are in the same slot
SET {order:5000}.details '{"item":"widget","qty":3}'
SET {order:5000}.payment '{"method":"card","status":"paid"}'
SET {order:5000}.shipping '{"carrier":"fedex","tracking":"FX123"}'

# Multi-key operations work because all keys share the {order:5000} hash tag
MGET {order:5000}.details {order:5000}.payment {order:5000}.shipping

# Transaction across same-slot keys
MULTI
SET {order:5000}.details '{"item":"widget","qty":3,"status":"confirmed"}'
SET {order:5000}.payment '{"method":"card","status":"captured"}'
EXEC

用于 Kubernetes 的 Redis 操作器

在 Kubernetes 中运行 Redis 需要仔细处理持久存储、网络身份、优雅的故障转移和配置管理。 Kubernetes 操作员将此操作知识编码到自定义控制器中，这些控制器通过自定义资源定义 (CRDs) 以声明方式管理 Redis 集群。

Spotahome Redis 操作员

Spotahome 运算符（也称为 redis-operator）是一个成熟的、广泛使用的运算符，用于在 Kubernetes 中部署基于 Redis Sentinel 的 HA。它通过基于 Sentinel 的故障转移来管理 Redis 主副本集。

# Install the Spotahome Redis Operator
helm repo add spotahome https://spotahome.github.io/redis-operator
helm install redis-operator spotahome/redis-operator \
  --namespace redis-system --create-namespace

# RedisFailover CRD — 3 Redis instances + 3 Sentinels
apiVersion: databases.spotahome.com/v1
kind: RedisFailover
metadata:
  name: redis-ha
  namespace: production
spec:
  sentinel:
    replicas: 3
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 200m
        memory: 256Mi
    customConfig:
      down-after-milliseconds: "5000"
      failover-timeout: "60000"
  redis:
    replicas: 3
    resources:
      requests:
        cpu: "2"
        memory: 8Gi
      limits:
        cpu: "4"
        memory: 16Gi
    storage:
      persistentVolumeClaim:
        metadata:
          name: redis-data
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 50Gi
          storageClassName: longhorn
    customConfig:
      maxmemory: "6gb"
      maxmemory-policy: "allkeys-lru"
      save: "900 1 300 10 60 10000"
      appendonly: "yes"
      appendfsync: "everysec"
      aof-use-rdb-preamble: "yes"
      repl-backlog-size: "256mb"
    exporter:
      enabled: true
      image: oliver006/redis_exporter:latest
      args:
        - --include-system-metrics

OpsTree Redis 操作器

OpsTree 运算符支持 Redis Sentinel（独立 HA）和 Redis Cluster（分片 HA）拓扑，使其更适合不同的用例。

# Install the OpsTree Redis Operator
helm repo add ot-helm https://ot-container-kit.github.io/helm-charts/
helm install redis-operator ot-helm/redis-operator \
  --namespace redis-system --create-namespace

# Redis Cluster CRD — 3 masters + 3 replicas
apiVersion: redis.redis.opstreelabs.in/v1beta2
kind: RedisCluster
metadata:
  name: redis-cluster
  namespace: production
spec:
  clusterSize: 3
  clusterVersion: v7
  persistenceEnabled: true
  kubernetesConfig:
    image: redis:7.2-alpine
    imagePullPolicy: IfNotPresent
    resources:
      requests:
        cpu: "2"
        memory: 8Gi
      limits:
        cpu: "4"
        memory: 16Gi
  redisLeader:
    replicas: 3
    redisConfig:
      additionalRedisConfig: |
        maxmemory 6gb
        maxmemory-policy allkeys-lru
        appendonly yes
        appendfsync everysec
  redisFollower:
    replicas: 3
    redisConfig:
      additionalRedisConfig: |
        maxmemory 6gb
        replica-read-only yes
  storage:
    volumeClaimTemplate:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 50Gi
        storageClassName: longhorn
  redisExporter:
    enabled: true
    image: quay.io/opstree/redis-exporter:v1.44.0

Redis 企业运营商

Redis Enterprise 为商业 Kubernetes 运营商提供先进功能，包括主动-主动异地复制 (CRDT)、Redis 模块支持、自动分层（RAM + 闪存）和自动化集群管理。对于需要企业级 SLA 和支持的组织来说，这是推荐的选项。

# Redis Enterprise Operator CRD
apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: redis-enterprise
  namespace: redis-enterprise
spec:
  nodes: 3
  persistentSpec:
    enabled: true
    storageClassName: longhorn
    volumeSize: 100Gi
  redisEnterpriseNodeResources:
    limits:
      cpu: "8"
      memory: 32Gi
    requests:
      cpu: "4"
      memory: 16Gi
  uiServiceType: ClusterIP
  servicesRiggerSpec:
    databaseServiceType: ClusterIP
---
apiVersion: app.redislabs.com/v1alpha1
kind: RedisEnterpriseDatabase
metadata:
  name: redis-ha-db
  namespace: redis-enterprise
spec:
  memorySize: 10GB
  replication: true
  shardCount: 3
  persistence: aofEverySecond
  tlsMode: enabled
  modulesList:
    - name: search
      version: latest
    - name: json
      version: latest

持久化策略：RDB、AOF 和混合

Redis 提供三种持久机制。选择正确的策略取决于您的恢复点目标 (RPO)、性能要求和存储限制。

RDB 快照按配置的时间间隔创建整个数据集的时间点快照。它们结构紧凑，重启时加载速度快，非常适合备份。但是，快照之间写入的数据会在崩溃时丢失。 RDB 使用分叉子进程，因此快照创建不会阻塞主 Redis 线程，但分叉本身可能会由于写入时复制内存分配而导致大型数据集出现延迟峰值。

AOF（仅附加文件）记录对磁盘的每个写入操作。它提供了比 RDB 更好的耐用性 — 使用appendfsync everysec，您在崩溃时最多会丢失一秒的数据。使用appendfsync always，您不会有任何损失，但会付出巨大的性能代价。 AOF 文件比 RDB 文件大，并且重启时加载速度较慢。

混合持久性（推荐方法）结合了两者：aof-use-rdb-preamble yes在 AOF 文件的开头写入 RDB 快照，然后是用于后续写入的 AOF 条目。这提供了快速的启动时间（RDB 部分）和强大的耐用性（AOF 部分）。

# redis.conf — Hybrid persistence (recommended for production)

# RDB snapshots
save 900 1          # Snapshot if at least 1 write in 900 seconds
save 300 10         # Snapshot if at least 10 writes in 300 seconds
save 60 10000       # Snapshot if at least 10000 writes in 60 seconds
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /data/redis

# AOF
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec           # Best balance of durability and performance
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-use-rdb-preamble yes       # Hybrid: RDB preamble + AOF tail
aof-timestamp-enabled yes      # Enable timestamps for PITR (Redis 7+)

# Recovery options
rdb-del-sync-files no
aof-load-truncated yes

云管理的 Redis 部署

AWS 适用于 Redis 的 ElastiCache

AWS ElastiCache 提供完全托管的 Redis，具有两种 HA 模式：禁用集群模式（单分片，最多 5 个副本，类似 Sentinel 的故障转移）和启用集群模式（最多 500 个分片，每个分片最多 5 个副本，哈希槽分配）。全局数据存储提供跨区域复制以实现灾难恢复。

# AWS CLI — Create ElastiCache Redis Cluster Mode Enabled
aws elasticache create-replication-group \
  --replication-group-id redis-ha-prod \
  --replication-group-description "Production Redis HA Cluster" \
  --engine redis \
  --engine-version 7.1 \
  --cache-node-type cache.r7g.2xlarge \
  --num-node-groups 3 \
  --replicas-per-node-group 2 \
  --automatic-failover-enabled \
  --multi-az-enabled \
  --at-rest-encryption-enabled \
  --transit-encryption-enabled \
  --auth-token strong_auth_token \
  --cache-subnet-group-name redis-subnet-group \
  --security-group-ids sg-0123456789abcdef0 \
  --snapshot-retention-limit 7 \
  --snapshot-window "03:00-05:00" \
  --preferred-maintenance-window "sun:05:00-sun:07:00" \
  --cache-parameter-group-name redis-ha-params \
  --log-delivery-configurations '[
    {"LogType":"slow-log","DestinationType":"cloudwatch-logs","DestinationDetails":{"CloudWatchLogsDetails":{"LogGroup":"/aws/elasticache/redis-ha-prod"}}},
    {"LogType":"engine-log","DestinationType":"cloudwatch-logs","DestinationDetails":{"CloudWatchLogsDetails":{"LogGroup":"/aws/elasticache/redis-ha-prod"}}}
  ]'

# Create Global Datastore for cross-region DR
aws elasticache create-global-replication-group \
  --global-replication-group-id-suffix redis-global \
  --primary-replication-group-id redis-ha-prod

# Add secondary region
aws elasticache create-replication-group \
  --replication-group-id redis-ha-dr \
  --replication-group-description "DR Redis in eu-west-2" \
  --global-replication-group-id ldgnf-redis-global \
  --cache-node-type cache.r7g.2xlarge \
  --num-node-groups 3 \
  --replicas-per-node-group 1 \
  --region eu-west-2

# Custom parameter group for HA tuning
aws elasticache create-cache-parameter-group \
  --cache-parameter-group-name redis-ha-params \
  --cache-parameter-group-family redis7 \
  --description "HA-optimised Redis 7 parameters"

aws elasticache modify-cache-parameter-group \
  --cache-parameter-group-name redis-ha-params \
  --parameter-name-values \
    "ParameterName=maxmemory-policy,ParameterValue=allkeys-lru" \
    "ParameterName=timeout,ParameterValue=300" \
    "ParameterName=tcp-keepalive,ParameterValue=60" \
    "ParameterName=activedefrag,ParameterValue=yes"

用于 Redis 的 Azure 缓存

Azure 缓存提供三层：基本（无复制）、标准（复制）和高级/企业级。高级层支持集群、异地复制、区域冗余、VNet 注入和数据持久性。企业层添加了 Redis 模块和主动-主动地理分布。

# Azure CLI — Create Premium Azure Cache for Redis with clustering
az redis create \
  --resource-group redis-ha-rg \
  --name redis-ha-prod \
  --location westeurope \
  --sku Premium \
  --vm-size P3 \
  --shard-count 3 \
  --replicas-per-master 1 \
  --zones 1 2 3 \
  --minimum-tls-version 1.2 \
  --redis-version 7

# Enable geo-replication (link primary to secondary)
az redis server-link create \
  --name redis-ha-prod \
  --resource-group redis-ha-rg \
  --server-to-link /subscriptions/.../redis-ha-dr \
  --replication-role Secondary

# Configure data persistence
az redis update \
  --name redis-ha-prod \
  --resource-group redis-ha-rg \
  --set redisConfiguration.rdb-backup-enabled=true \
  --set redisConfiguration.rdb-backup-frequency=60 \
  --set redisConfiguration.rdb-storage-connection-string="DefaultEndpointsProtocol=https;..."

# Enable diagnostics
az monitor diagnostic-settings create \
  --name redis-diagnostics \
  --resource /subscriptions/.../redis-ha-prod \
  --workspace /subscriptions/.../log-analytics-workspace \
  --metrics '[{"category":"AllMetrics","enabled":true}]'

GCP 内存存储

GCP 提供两个 Memorystore 层：标准（具有自动故障转移副本的单实例）和Redis 集群（具有自动扩展功能的分片、完全托管集群）。标准层适用于大多数 HA 使用案例，而 Redis Cluster 则处理需要水平扩展的大型数据集。

# GCP — Create Standard tier Memorystore (HA with auto-failover)
gcloud redis instances create redis-ha-prod \
  --size=26 \
  --region=europe-west1 \
  --zone=europe-west1-b \
  --alternative-zone=europe-west1-c \
  --tier=standard \
  --redis-version=redis_7_2 \
  --redis-config="maxmemory-policy=allkeys-lru,activedefrag=yes" \
  --network=projects/my-project/global/networks/vpc-main \
  --transit-encryption-mode=SERVER_AUTHENTICATION \
  --enable-auth \
  --persistence-mode=RDB \
  --rdb-snapshot-period=12h \
  --rdb-snapshot-start-time="2026-04-12T03:00:00Z" \
  --maintenance-window-day=SUNDAY \
  --maintenance-window-hour=4

# GCP — Create Memorystore Redis Cluster
gcloud redis clusters create redis-cluster-prod \
  --region=europe-west1 \
  --shard-count=3 \
  --replica-count=1 \
  --network=projects/my-project/global/networks/vpc-main \
  --transit-encryption-mode=SERVER_AUTHENTICATION

多区域 Redis 部署

多区域 Redis 部署对于灾难恢复和减少全球延迟至关重要。该方法因架构而异：Redis Enterprise 使用主动-主动与 CRDT（无冲突复制数据类型）实现真正的多主写入，而开源 Redis 和云托管服务则使用主动-被动复制与辅助区域中的只读副本。

裸机 k3s/带 Longhorn

对于运行自己的硬件的组织来说，在裸机 k3s 上部署 Redis HA 可以提供对基础设施的完全控制，消除云供应商锁定，并且对于大规模部署来说可以显着提高成本效益。 k3s 是经过认证的轻量级 Kubernetes 发行版，非常适合边缘和裸机环境，而 Rancher 则提供了用于多集群操作的管理平面。

k3s 安装和 Redis 操作员部署

# Install k3s on the first server node
curl -sfL https://get.k3s.io | K3S_TOKEN=redis-cluster-token \
  INSTALL_K3S_EXEC="server --cluster-init --disable traefik --disable servicelb" sh -

# Join additional server nodes
curl -sfL https://get.k3s.io | K3S_TOKEN=redis-cluster-token \
  K3S_URL=https://10.0.0.1:6443 \
  INSTALL_K3S_EXEC="server" sh -

# Install Longhorn for distributed block storage
helm repo add longhorn https://charts.longhorn.io
helm install longhorn longhorn/longhorn \
  --namespace longhorn-system --create-namespace \
  --set defaultSettings.defaultDataPath=/mnt/longhorn \
  --set defaultSettings.replicaCount=3 \
  --set defaultSettings.storageMinimalAvailablePercentage=15

# Install MetalLB for bare-metal LoadBalancer services
helm repo add metallb https://metallb.github.io/metallb
helm install metallb metallb/metallb --namespace metallb-system --create-namespace

# Configure MetalLB IP address pool
kubectl apply -f - <

Redis HA Helm k3s 的值

# values-redis-ha.yaml — Helm values for Redis HA on k3s/Longhorn
redis:
  replicas: 3
  resources:
    requests:
      cpu: "2"
      memory: 8Gi
    limits:
      cpu: "4"
      memory: 16Gi
  storage:
    persistentVolumeClaim:
      metadata:
        name: redis-data
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 50Gi
        storageClassName: longhorn
  customConfig:
    maxmemory: "6gb"
    maxmemory-policy: "allkeys-lru"
    save: "900 1 300 10 60 10000"
    appendonly: "yes"
    appendfsync: "everysec"
    aof-use-rdb-preamble: "yes"
    repl-backlog-size: "256mb"
    tcp-keepalive: "60"
    timeout: "300"
    hz: "10"
    activedefrag: "yes"
  exporter:
    enabled: true
    image: oliver006/redis_exporter:latest

sentinel:
  replicas: 3
  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      cpu: 200m
      memory: 256Mi
  customConfig:
    down-after-milliseconds: "5000"
    failover-timeout: "60000"
    parallel-syncs: "1"

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
            - key: app.kubernetes.io/component
              operator: In
              values:
                - redis
        topologyKey: kubernetes.io/hostname

内存管理和逐出策略

Redis 将所有数据存储在内存中，使内存管理成为最关键的操作问题。当 Redis 达到配置的maxmemory限制时，它必须决定如何处理传入的写入命令。驱逐政策控制这种行为。

# redis.conf — Memory management
maxmemory 6gb
maxmemory-policy allkeys-lru

# Available eviction policies:
# noeviction        — Return errors on writes when memory limit reached
# allkeys-lru       — Evict least recently used keys (general-purpose cache)
# allkeys-lfu       — Evict least frequently used keys (better for skewed access patterns)
# volatile-lru      — Evict LRU keys with TTL set
# volatile-lfu      — Evict LFU keys with TTL set
# volatile-ttl      — Evict keys with shortest TTL first
# allkeys-random    — Evict random keys
# volatile-random   — Evict random keys with TTL set

# Active defragmentation (Redis 4.0+)
activedefrag yes
active-defrag-enabled yes
active-defrag-ignore-bytes 100mb
active-defrag-threshold-lower 10
active-defrag-threshold-upper 100
active-defrag-cycle-min 1
active-defrag-cycle-max 25
active-defrag-max-scan-fields 1000

# Memory usage monitoring
# redis-cli INFO memory
# Key metrics:
#   used_memory           — Total bytes allocated by Redis
#   used_memory_rss       — Resident set size (OS-level memory)
#   mem_fragmentation_ratio — RSS / used_memory (should be close to 1.0)
#   maxmemory             — Configured memory limit
#   evicted_keys          — Total keys evicted due to maxmemory

对于 HA 部署，将maxmemory设置为节点可用 RAM 的大约 75%。剩余的 25% 用于容纳复制输出缓冲区、AOF 重写缓冲区、RDB 快照期间的写时复制内存以及操作系统开销。对于 16 GB 的 Pod，将 maxmemory 设置为 12 GB。 对于 64GB 裸机服务器，将其设置为 48GB。

TLS 加密和 ACL

生产 Redis 部署必须使用 TLS 加密传输中的数据，并使用 Redis 6.0 中引入的 ACL（访问控制列表）实施细粒度访问控制。

# redis.conf — TLS configuration
tls-port 6380
port 0                            # Disable non-TLS port entirely
tls-cert-file /etc/redis/tls/redis.crt
tls-key-file /etc/redis/tls/redis.key
tls-ca-cert-file /etc/redis/tls/ca.crt
tls-auth-clients optional         # Require client certificates (mutual TLS)
tls-replication yes               # Encrypt replication traffic
tls-cluster yes                   # Encrypt cluster bus traffic
tls-protocols "TLSv1.3"           # Only allow TLS 1.3

# ACL configuration (Redis 6.0+)
# user  on|off [>password] [~pattern] [+command|-command] [&channel]
user default off                  # Disable the default user
user admin on >strong_admin_pass ~* +@all
user appuser on >app_pass ~app:* ~session:* ~cache:* +@read +@write +@connection -@admin -@dangerous
user readonly on >readonly_pass ~* +@read +@connection -@write -@admin
user replicator on >repl_pass +psync +replconf +ping

# Load ACL from external file
aclfile /etc/redis/users.acl

# Redis Streams with consumer groups — HA-safe message processing

# Create a stream and consumer group
XGROUP CREATE events:orders orders-processors $ MKSTREAM

# Produce events
XADD events:orders * action "order_placed" order_id "12345" amount "599.99"
XADD events:orders * action "order_placed" order_id "12346" amount "149.99"

# Consume events (in consumer group — at-least-once delivery)
XREADGROUP GROUP orders-processors worker-1 COUNT 10 BLOCK 5000 STREAMS events:orders >

# Acknowledge processed events
XACK events:orders orders-processors 1681234567890-0

# Check pending messages (unacknowledged)
XPENDING events:orders orders-processors - + 10

# Claim abandoned messages (from a dead consumer)
XAUTOCLAIM events:orders orders-processors worker-2 60000 0-0 COUNT 10

# Trim stream to prevent unbounded growth
XTRIM events:orders MAXLEN ~ 100000

# redis.conf — Loading modules
loadmodule /opt/redis-stack/lib/rejson.so
loadmodule /opt/redis-stack/lib/redisearch.so
loadmodule /opt/redis-stack/lib/redistimeseries.so

# Modules are replicated to replicas via the command stream
# Ensure the same modules are installed on all nodes (master + replicas)

# RedisJSON — store and query JSON documents
JSON.SET user:1001 $ '{"name":"Alice","email":"alice@example.com","orders":42}'
JSON.GET user:1001 $.name

# RediSearch — full-text search with indexing
FT.CREATE idx:users ON JSON PREFIX 1 user: SCHEMA $.name AS name TEXT $.email AS email TAG $.orders AS orders NUMERIC
FT.SEARCH idx:users "@name:Alice"

# RedisTimeSeries — time-series data
TS.CREATE metrics:cpu:node1 RETENTION 86400000 LABELS host node1 metric cpu
TS.ADD metrics:cpu:node1 * 73.5
TS.RANGE metrics:cpu:node1 - + AGGREGATION avg 60000

# Node.js — ioredis connection pool with cluster mode
const Redis = require('ioredis');

// Cluster mode connection
const cluster = new Redis.Cluster(
  [
    { host: '10.0.1.10', port: 7000 },
    { host: '10.0.1.11', port: 7000 },
    { host: '10.0.1.12', port: 7000 }
  ],
  {
    redisOptions: {
      password: 'cluster_password',
      connectTimeout: 10000,
      maxRetriesPerRequest: 3
    },
    scaleReads: 'slave',           // Route reads to replicas
    clusterRetryStrategy(times) {
      return Math.min(times * 200, 5000);
    },
    slotsRefreshTimeout: 2000,
    slotsRefreshInterval: 5000,
    enableOfflineQueue: true,
    enableReadyCheck: true,
    natMap: {}                     // For NAT/port-forwarded environments
  }
);

cluster.on('error', (err) => console.error('Cluster error:', err));
cluster.on('node error', (err, address) => {
  console.error(`Node ${address} error:`, err);
});

# Python — redis-py connection pool with cluster mode
from redis.cluster import RedisCluster
from redis.backoff import ExponentialBackoff
from redis.retry import Retry

retry = Retry(ExponentialBackoff(cap=5, base=0.1), retries=5)

rc = RedisCluster(
    startup_nodes=[
        {"host": "10.0.1.10", "port": 7000},
        {"host": "10.0.1.11", "port": 7000},
        {"host": "10.0.1.12", "port": 7000}
    ],
    password="cluster_password",
    decode_responses=True,
    read_from_replicas=True,
    retry=retry,
    retry_on_timeout=True,
    socket_timeout=5,
    socket_connect_timeout=5,
    max_connections=50,
    health_check_interval=30
)

rc.set("key", "value")
print(rc.get("key"))

// Go — go-redis cluster client with connection pooling
package main

import (
    "context"
    "time"

    "github.com/redis/go-redis/v9"
)

func NewRedisCluster() *redis.ClusterClient {
    return redis.NewClusterClient(&redis.ClusterOptions{
        Addrs: []string{
            "10.0.1.10:7000",
            "10.0.1.11:7000",
            "10.0.1.12:7000",
        },
        Password:         "cluster_password",
        ReadOnly:         true,
        RouteRandomly:    true,
        RouteByLatency:   false,
        PoolSize:         50,
        MinIdleConns:     10,
        DialTimeout:      10 * time.Second,
        ReadTimeout:      5 * time.Second,
        WriteTimeout:     5 * time.Second,
        PoolTimeout:      10 * time.Second,
        MaxRetries:       5,
        MinRetryBackoff:  200 * time.Millisecond,
        MaxRetryBackoff:  5 * time.Second,
    })
}

#!/bin/bash
# redis-backup.sh — Automated Redis backup script

REDIS_HOST="10.0.1.10"
REDIS_PORT="6379"
REDIS_PASS="strong_master_password"
BACKUP_DIR="/backups/redis"
S3_BUCKET="s3://redis-backups-prod"
RETENTION_DAYS=30
DATE=$(date +%Y%m%d_%H%M%S)

mkdir -p "$BACKUP_DIR"

# Trigger RDB snapshot on the master
redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASS" BGSAVE

# Wait for background save to complete
while [ "$(redis-cli -h $REDIS_HOST -p $REDIS_PORT -a $REDIS_PASS LASTSAVE)" = "$(redis-cli -h $REDIS_HOST -p $REDIS_PORT -a $REDIS_PASS LASTSAVE)" ]; do
  sleep 1
done
sleep 2

# Copy the RDB file
RDB_FILE=$(redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASS" CONFIG GET dir | tail -1)
cp "${RDB_FILE}/dump.rdb" "${BACKUP_DIR}/dump_${DATE}.rdb"

# Compress and upload to S3
gzip "${BACKUP_DIR}/dump_${DATE}.rdb"
aws s3 cp "${BACKUP_DIR}/dump_${DATE}.rdb.gz" "${S3_BUCKET}/daily/dump_${DATE}.rdb.gz" \
  --storage-class STANDARD_IA

# Cleanup old local backups
find "$BACKUP_DIR" -name "dump_*.rdb.gz" -mtime +$RETENTION_DAYS -delete

# Verify backup integrity
redis-check-rdb "${BACKUP_DIR}/dump_${DATE}.rdb.gz" && \
  echo "Backup verified: dump_${DATE}.rdb.gz" || \
  echo "ERROR: Backup verification failed!"

echo "Backup complete: ${BACKUP_DIR}/dump_${DATE}.rdb.gz"

# Restore from RDB backup
# 1. Stop Redis
systemctl stop redis

# 2. Replace the RDB file
gunzip /backups/redis/dump_20260412_030000.rdb.gz
cp /backups/redis/dump_20260412_030000.rdb /data/redis/dump.rdb
chown redis:redis /data/redis/dump.rdb

# 3. Disable AOF temporarily (if enabled) to prevent AOF overriding RDB on startup
redis-cli -a password CONFIG SET appendonly no

# 4. Start Redis (loads RDB)
systemctl start redis

# 5. Re-enable AOF and rewrite it from the loaded data
redis-cli -a password CONFIG SET appendonly yes
redis-cli -a password BGREWRITEAOF

# Key Redis INFO sections for HA monitoring
redis-cli -a password INFO replication
# role:master
# connected_slaves:2
# slave0:ip=10.0.1.11,port=6379,state=online,offset=1234567,lag=0
# slave1:ip=10.0.1.12,port=6379,state=online,offset=1234560,lag=1
# master_replid:abc123...
# master_repl_offset:1234567
# repl_backlog_size:268435456
# repl_backlog_first_byte_offset:1000000

redis-cli -a password INFO memory
# used_memory:6442450944
# used_memory_human:6.00G
# used_memory_rss:6879707136
# mem_fragmentation_ratio:1.07
# maxmemory:6442450944
# maxmemory_policy:allkeys-lru
# evicted_keys:12345

redis-cli -a password INFO stats
# total_connections_received:50000
# total_commands_processed:12345678
# instantaneous_ops_per_sec:85432
# keyspace_hits:11000000
# keyspace_misses:1345678
# expired_keys:500000
# evicted_keys:12345

redis-cli -a password INFO clients
# connected_clients:150
# blocked_clients:0
# maxclients:10000

# Prometheus Redis Exporter deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-exporter
  namespace: monitoring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis-exporter
  template:
    metadata:
      labels:
        app: redis-exporter
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "9121"
    spec:
      containers:
        - name: redis-exporter
          image: oliver006/redis_exporter:latest
          args:
            - --redis.addr=redis://redis-ha-master:6379
            - --redis.password=$(REDIS_PASSWORD)
            - --include-system-metrics
            - --is-cluster
          env:
            - name: REDIS_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: redis-secret
                  key: password
          ports:
            - containerPort: 9121
              name: metrics
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 200m
              memory: 256Mi

# PrometheusRule for Redis HA alerts
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: redis-ha-alerts
  namespace: monitoring
spec:
  groups:
    - name: redis-availability
      rules:
        - alert: RedisDown
          expr: redis_up == 0
          for: 1m
          labels:
            severity: critical
          annotations:
            summary: "Redis instance {{ $labels.instance }} is down"

        - alert: RedisReplicaDisconnected
          expr: redis_connected_slaves < 2
          for: 2m
          labels:
            severity: warning
          annotations:
            summary: "Redis master has fewer than 2 connected replicas"

        - alert: RedisReplicationLagHigh
          expr: redis_replication_lag > 5
          for: 3m
          labels:
            severity: warning
          annotations:
            summary: "Redis replication lag exceeds 5 seconds on {{ $labels.instance }}"

        - alert: RedisMemoryUsageHigh
          expr: redis_memory_used_bytes / redis_memory_max_bytes > 0.9
          for: 5m
          labels:
            severity: warning
          annotations:
            summary: "Redis memory usage above 90% on {{ $labels.instance }}"

        - alert: RedisEvictionsHigh
          expr: rate(redis_evicted_keys_total[5m]) > 100
          for: 5m
          labels:
            severity: warning
          annotations:
            summary: "Redis evicting keys at >100/s on {{ $labels.instance }}"

        - alert: RedisKeyspaceHitRateLow
          expr: redis_keyspace_hits_total / (redis_keyspace_hits_total + redis_keyspace_misses_total) < 0.8
          for: 10m
          labels:
            severity: info
          annotations:
            summary: "Redis cache hit rate below 80% on {{ $labels.instance }}"

        - alert: RedisSentinelDown
          expr: redis_sentinel_master_status != 1
          for: 1m
          labels:
            severity: critical
          annotations:
            summary: "Redis Sentinel reports master unhealthy"

    - name: redis-performance
      rules:
        - alert: RedisSlowlogGrowing
          expr: increase(redis_slowlog_length[5m]) > 10
          for: 5m
          labels:
            severity: warning
          annotations:
            summary: "Redis slow log growing rapidly on {{ $labels.instance }}"

        - alert: RedisConnectionsNearLimit
          expr: redis_connected_clients / redis_config_maxclients > 0.8
          for: 5m
          labels:
            severity: warning
          annotations:
            summary: "Redis connected clients above 80% of maxclients"

# Run Dragonfly as a Redis replacement
docker run -d --name dragonfly \
  -p 6379:6379 \
  -v /data/dragonfly:/data \
  docker.dragonflydb.io/dragonflydb/dragonfly \
  --maxmemory 12gb \
  --proactor_threads 8 \
  --dbfilename dump.rdb \
  --requirepass strong_password

# Dragonfly in Kubernetes
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: dragonfly
  namespace: production
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dragonfly
  template:
    metadata:
      labels:
        app: dragonfly
    spec:
      containers:
        - name: dragonfly
          image: docker.dragonflydb.io/dragonflydb/dragonfly:latest
          args:
            - --maxmemory=12gb
            - --proactor_threads=8
            - --requirepass=strong_password
            - --snapshot_cron=*/30 * * * *
          ports:
            - containerPort: 6379
          resources:
            requests:
              cpu: "4"
              memory: 16Gi
            limits:
              cpu: "8"
              memory: 16Gi
          volumeMounts:
            - name: data
              mountPath: /data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: longhorn
        resources:
          requests:
            storage: 100Gi

# keydb.conf — Multi-threaded configuration with active replication
server-threads 4                  # Use 4 threads for command processing
bind 0.0.0.0
port 6379
requirepass strong_password
masterauth strong_password

# Active-active replication (multi-master)
active-replica yes
replicaof peer-host 6379          # Bidirectional replication

# On the peer node, configure the reverse:
# replicaof this-host 6379

# FLASH storage tiering (for datasets larger than RAM)
# storage-provider flash /mnt/flash-storage 100
# maxmemory 16gb
# Will keep hot data in RAM and spill cold data to SSD

# SubKey expiration (unique to KeyDB)
# Allows setting TTL on hash fields, not just top-level keys
# EXPIREMEMBER myhash field1 3600

# Python — Pipelining with redis-py
import redis
import time

r = redis.Redis(host='10.0.1.10', port=6379, password='password', decode_responses=True)

# Without pipelining: 1000 round-trips
start = time.time()
for i in range(1000):
    r.set(f'key:{i}', f'value:{i}')
print(f'Without pipeline: {time.time() - start:.3f}s')

# With pipelining: 1 round-trip for 1000 commands
start = time.time()
pipe = r.pipeline(transaction=False)
for i in range(1000):
    pipe.set(f'key:{i}', f'value:{i}')
pipe.execute()
print(f'With pipeline: {time.time() - start:.3f}s')
# Typically 5-10x faster

# Lua script for atomic rate limiting
# KEYS[1] = rate limit key
# ARGV[1] = max requests
# ARGV[2] = window in seconds
local current = redis.call('INCR', KEYS[1])
if current == 1 then
    redis.call('EXPIRE', KEYS[1], ARGV[2])
end
if current > tonumber(ARGV[1]) then
    return 0  -- Rate limited
end
return 1  -- Allowed

# Load and execute
redis-cli -a password EVAL "\
  local current = redis.call('INCR', KEYS[1]) \
  if current == 1 then redis.call('EXPIRE', KEYS[1], ARGV[2]) end \
  if current > tonumber(ARGV[1]) then return 0 end \
  return 1" 1 ratelimit:user:123 100 60

# redis.conf — Memory optimisation settings

# Use ziplist encoding for small hashes, lists, sorted sets
hash-max-listpack-entries 128
hash-max-listpack-value 64
list-max-listpack-size -2          # 8KB per node
zset-max-listpack-entries 128
zset-max-listpack-value 64
set-max-intset-entries 512

# Lazy freeing (avoid blocking on large key deletion)
lazyfree-lazy-eviction yes
lazyfree-lazy-expire yes
lazyfree-lazy-server-del yes
lazyfree-lazy-user-del yes
lazyfree-lazy-user-flush yes

# jemalloc tuning
# MALLOC_CONF="background_thread:true,dirty_decay_ms:5000,muzzy_decay_ms:5000"

# Analyse memory usage
redis-cli -a password MEMORY DOCTOR
redis-cli -a password MEMORY STATS
redis-cli -a password --bigkeys
redis-cli -a password --memkeys

# Diagnosis
redis-cli -p 26379 SENTINEL master mymaster
# Check: flags should show 's_down' or 'o_down' if master is unreachable

redis-cli -p 26379 SENTINEL get-master-addr-by-name mymaster
# Returns the current master address (should be the promoted replica)

# Check Sentinel logs for failover events
tail -f /var/log/redis/sentinel.log
# Look for: +sdown, +odown, +try-failover, +elected-leader,
#           +failover-state-select-slave, +selected-slave,
#           +failover-state-send-slaveof-noone, +failover-end

# Diagnosis: check if min-replicas-to-write is configured
redis-cli -a password CONFIG GET min-replicas-to-write
redis-cli -a password CONFIG GET min-replicas-max-lag

# Prevention: configure min-replicas on all masters
redis-cli -a password CONFIG SET min-replicas-to-write 1
redis-cli -a password CONFIG SET min-replicas-max-lag 10

# If split-brain occurred: identify the stale master
# Compare replication offsets — the master with the higher offset has more data
redis-cli -h master1-ip -a password INFO replication | grep master_repl_offset
redis-cli -h master2-ip -a password INFO replication | grep master_repl_offset

# Force the stale master to become a replica
redis-cli -h stale-master-ip -a password REPLICAOF correct-master-ip 6379

# Diagnosis: check replication backlog
redis-cli -a password INFO replication | grep repl_backlog
# If repl_backlog_first_byte_offset is ahead of replica's offset, full sync triggers

# Prevention: increase backlog size
redis-cli -a password CONFIG SET repl-backlog-size 512mb

# Monitor for full syncs
redis-cli -a password INFO stats | grep sync_full
redis-cli -a password INFO stats | grep sync_partial_ok
redis-cli -a password INFO stats | grep sync_partial_err

# Diagnosis
redis-cli -a password INFO memory
# Check: used_memory vs maxmemory, mem_fragmentation_ratio

redis-cli -a password MEMORY DOCTOR
# Returns advice on memory issues

# Prevention: set proper maxmemory and eviction
redis-cli -a password CONFIG SET maxmemory 12gb
redis-cli -a password CONFIG SET maxmemory-policy allkeys-lru

# Find large keys consuming memory
redis-cli -a password --bigkeys
redis-cli -a password --memkeys --memkeys-samples 100

# Emergency: manually evict keys
redis-cli -a password SCAN 0 COUNT 1000 TYPE string
# Identify and DEL unnecessary large keys

# Diagnosis: check slowlog
redis-cli -a password SLOWLOG GET 20
redis-cli -a password SLOWLOG LEN

# Check for blocking commands
redis-cli -a password CLIENT LIST | grep -E 'cmd=(keys|sort|smembers)'

# Prevention: configure slowlog threshold
redis-cli -a password CONFIG SET slowlog-log-slower-than 10000   # 10ms
redis-cli -a password CONFIG SET slowlog-max-len 256

# Rename dangerous commands
rename-command KEYS ""              # Disable KEYS entirely
rename-command FLUSHALL ""          # Disable FLUSHALL
rename-command FLUSHDB ""           # Disable FLUSHDB
rename-command DEBUG ""             # Disable DEBUG

# Capacity estimation formulas

# Memory per node:
#   Base dataset size (use redis-cli DBSIZE and MEMORY USAGE on a sample)
#   + Replication output buffer: ~64MB per replica
#   + AOF rewrite buffer: ~64MB during rewrites
#   + Copy-on-write overhead during BGSAVE: up to 2x during heavy writes
#   + Client output buffers: ~1KB per client
#   + OS overhead: ~1-2GB
#   Rule of thumb: maxmemory = 75% of available RAM

# Network bandwidth:
#   Replication: write_throughput_bytes * num_replicas
#   Client traffic: ops_per_sec * avg_response_size
#   Full sync: dataset_size (one-time during replica bootstrap or failover)

# Example sizing for 20GB dataset, 100K ops/sec:
#   RAM per node: 20GB data + 4GB buffers + 2GB OS = 26GB -> 32GB node (75% = 24GB maxmemory)
#   CPU: 1 core handles ~100K ops/sec for simple commands (GET/SET)
#   Network: 100K ops * 1KB avg = 100MB/s client + 50MB/s replication = 150MB/s per master

# Scaling decision tree:
#   Need more read throughput? -> Add replicas (up to 5 per master)
#   Need more write throughput? -> Redis Cluster (add shards)
#   Need more memory? -> Redis Cluster (distribute dataset across shards)
#   Need lower latency? -> Reduce network hops (co-locate, use unix sockets)
#   Need global distribution? -> Multi-region replication or Redis Enterprise Active-Active

# Add shards to an existing Redis Cluster

# 1. Start new Redis nodes
redis-server /etc/redis/new-master.conf
redis-server /etc/redis/new-replica.conf

# 2. Add the new master to the cluster
redis-cli --cluster add-node new-master:7000 existing-node:7000 -a password

# 3. Add the new replica to follow the new master
redis-cli --cluster add-node new-replica:7000 existing-node:7000 \
  --cluster-slave --cluster-master-id NEW_MASTER_ID -a password

# 4. Reshard slots to the new master
redis-cli --cluster reshard existing-node:7000 \
  --cluster-from all --cluster-to NEW_MASTER_ID \
  --cluster-slots 4096 --cluster-yes -a password

# 5. Verify the new slot distribution
redis-cli -c -h existing-node -p 7000 -a password CLUSTER SLOTS

# Remove a shard (scale down)
# 1. Reshard all slots away from the node
redis-cli --cluster reshard existing-node:7000 \
  --cluster-from REMOVING_NODE_ID --cluster-to TARGET_NODE_ID \
  --cluster-slots 5461 --cluster-yes -a password

# 2. Remove the empty node
redis-cli --cluster del-node existing-node:7000 REMOVING_NODE_ID -a password

# When to scale vertically vs horizontally:

# Scale UP (bigger instances) when:
#   - Dataset fits in single-node memory
#   - Workload uses multi-key operations (MGET, SUNION, Lua across keys)
#   - Operational simplicity is more important than cost efficiency
#   - Using Redis modules that don't support Cluster mode well

# Scale OUT (more shards) when:
#   - Dataset exceeds single-node memory
#   - Write throughput exceeds single-thread capacity (~200K ops/sec)
#   - You need per-shard isolation for multi-tenant workloads
#   - Cost per GB of RAM is a concern (many smaller nodes vs few large ones)

# Cloud instance recommendations:
# AWS:   cache.r7g.xlarge (4 vCPU, 26GB) to cache.r7g.16xlarge (64 vCPU, 419GB)
# Azure: P1 (6GB) to P5 (120GB) per shard
# GCP:   5GB to 300GB per instance (Memorystore Standard)

# Bare metal:
#   CPU: 2-4 cores dedicated to Redis (single-threaded, but background tasks use extra cores)
#   RAM: 32-128GB per node (NVMe for swap-as-last-resort)
#   Network: 10Gbps minimum, 25Gbps for large datasets
#   Storage: NVMe SSD for AOF/RDB persistence (IOPS matters for fsync)