第06章：生产环境部署：从原型到产品

监控系统、缓存策略、容量规划确保搜索系统稳定运行

📝 TL;DR (核心要点速览)
– 部署架构：主从复制 + 读写分离 + 负载均衡
– 性能调优：数据库参数优化 + 查询缓存 + 连接池配置
– 监控告警：实时性能监控 + 自动故障恢复 + 容量预警
– 运维策略：备份恢复 + 滚动更新 + 容量规划

1. 生产环境架构设计

1.1 从开发到生产的架构演进

开发环境：
┌─────────────────┐
│   单数据库实例   │
│  (索引+数据)     │
└─────────────────┘

测试环境：
┌─────────┐  ┌─────────┐
│ 主数据库 │ ← │ 从数据库 │
│ (写入)   │  │ (读取)   │
└─────────┘  └─────────┘

生产环境：
┌──────────────┐    ┌──────────────┐
│   负载均衡器   │ ← │   搜索网关     │
│   (Nginx)     │    │  (PHP-FPM)    │
└──────────────┘    └──────────────┘
       ↓                   ↓
┌──────────────┐    ┌──────────────┐
│   缓存层      │    │   搜索服务     │
│  (Redis)     │    │  (API)        │
└──────────────┘    └──────────────┘
       ↓                   ↓
┌─────────────────────────────────────┐
│          数据库集群                    │
│ ┌─────────┐  ┌─────────┐  ┌─────────┐ │
│ │ 主库    │←→│ 从库1   │←→│ 从库2   │ │
│ │ (写入)  │  │ (读取)  │  │ (读取)  │ │
│ └─────────┘  └─────────┘  └─────────┘ │
└─────────────────────────────────────┘

1.2 关键架构决策

数据库架构选择：

方案1：单库单实例 (开发/测试)
- 优点：简单、成本低
- 缺点：单点故障、性能瓶颈

方案2：主从复制 (小型生产)
- 优点：读写分离、故障转移
- 缺点：写性能有限、同步延迟

方案3：分库分表 (中型生产)
- 优点：水平扩展、性能提升
- 缺点：复杂度高、跨库查询困难

方案4：数据库集群 (大型生产)
- 优点：高可用、自动故障转移
- 缺点：成本高、配置复杂

缓存架构：

L1缓存：应用内存缓存 (5-10ms)
- 搜索结果缓存
- 查询模式缓存
- 用户偏好缓存

L2缓存：Redis分布式缓存 (10-20ms)
- 热点查询缓存
- 索引片段缓存
- 用户会话缓存

L3缓存：数据库查询缓存 (20-50ms)
- SQL查询结果缓存
- 预计算结果缓存
- 统计数据缓存

2. 数据库性能调优

2.1 MySQL生产环境配置

# my.cnf - 生产环境优化配置

[mysqld]
# === 基础配置 ===
server-id = 1
log-bin = mysql-bin
binlog-format = ROW
expire_logs_days = 7

# === 内存配置 ===
# InnoDB缓冲池大小：建议为物理内存的70-80%
innodb_buffer_pool_size = 16G
innodb_buffer_pool_instances = 8

# 查询缓存（MySQL 5.7+已废弃，但8.0之前的版本仍可用）
query_cache_type = 1
query_cache_size = 256M
query_cache_limit = 2M

# 连接配置
max_connections = 1000
max_connect_errors = 10000
wait_timeout = 28800
interactive_timeout = 28800

# === InnoDB优化 ===
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DIRECT
innodb_io_capacity = 2000
innodb_io_capacity_max = 4000

# 日志配置
innodb_log_file_size = 1G
innodb_log_buffer_size = 64M

# === 搜索相关优化 ===
# 全文搜索配置
ft_min_word_len = 2
ft_max_word_len = 84
ft_query_expansion_limit = 100

# 排序优化
sort_buffer_size = 4M
join_buffer_size = 4M

# 临时表配置
tmp_table_size = 256M
max_heap_table_size = 256M

# === 性能监控 ===
slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 0.5

# 慢查询记录
log_queries_not_using_indexes = 1
log_throttle_queries_not_using_indexes = 60

2.2 数据库连接池配置

class ProductionDatabasePool
{
    private array $masterConfig;
    private array $slaveConfigs;
    private array $connectionPools = [];
    private array $poolStats = [];

    public function __construct(array $config)
    {
        $this->masterConfig = $config['master'];
        $this->slaveConfigs = $config['slaves'];

        $this->initializeConnectionPools();
    }

    /**
     * 初始化连接池
     */
    private function initializeConnectionPools(): void
    {
        // 主库连接池（用于写入）
        $this->connectionPools['master'] = $this->createPool(
            $this->masterConfig,
            'master',
            20,  // 最大连接数
            5    // 最小连接数
        );

        // 从库连接池（用于读取）
        foreach ($this->slaveConfigs as $index => $config) {
            $this->connectionPools["slave_{$index}"] = $this->createPool(
                $config,
                "slave_{$index}",
                30,  // 读库可以更多连接
                10
            );
        }
    }

    /**
     * 创建连接池
     */
    private function createPool(array $config, string $name, int $maxSize, int $minSize): array
    {
        $pool = [
            'config' => $config,
            'connections' => [],
            'available' => [],
            'in_use' => [],
            'max_size' => $maxSize,
            'min_size' => $minSize,
            'created' => 0,
            'destroyed' => 0
        ];

        // 预创建最小连接数
        for ($i = 0; $i < $minSize; $i++) {
            $this->createConnection($pool, $name);
        }

        return $pool;
    }

    /**
     * 获取主库连接
     */
    public function getMasterConnection(): PDO
    {
        return $this->getConnection('master');
    }

    /**
     * 获取从库连接（负载均衡）
     */
    public function getSlaveConnection(): PDO
    {
        $slavePools = array_filter($this->connectionPools, function($key) {
            return strpos($key, 'slave_') === 0;
        }, ARRAY_FILTER_USE_KEY);

        $slaveKeys = array_keys($slavePools);
        $selectedSlave = $slaveKeys[array_rand($slaveKeys)];

        return $this->getConnection($selectedSlave);
    }

    /**
     * 从连接池获取连接
     */
    private function getConnection(string $poolName): PDO
    {
        $pool = &$this->connectionPools[$poolName];

        // 如果有可用连接，直接返回
        if (!empty($pool['available'])) {
            $connectionId = array_pop($pool['available']);
            $pool['in_use'][$connectionId] = true;
            return $this->validateConnection($pool['connections'][$connectionId]);
        }

        // 如果没有可用连接但还能创建新连接
        if ($pool['created'] < $pool['max_size']) {
            return $this->createConnection($pool, $poolName);
        }

        // 等待可用连接（带超时）
        $timeout = 5; // 5秒超时
        $startTime = time();

        while (empty($pool['available']) && (time() - $startTime) < $timeout) {
            usleep(100000); // 100ms
            if (!empty($pool['available'])) {
                $connectionId = array_pop($pool['available']);
                $pool['in_use'][$connectionId] = true;
                return $this->validateConnection($pool['connections'][$connectionId]);
            }
        }

        throw new RuntimeException("Connection pool exhausted for {$poolName}");
    }

    /**
     * 创建新连接
     */
    private function createConnection(array &$pool, string $poolName): PDO
    {
        $config = $pool['config'];
        $dsn = "mysql:host={$config['host']};port={$config['port']};dbname={$config['database']};charset=utf8mb4";

        $options = [
            PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
            PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC,
            PDO::ATTR_PERSISTENT => false,
            PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci",
            PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => true,
            PDO::ATTR_TIMEOUT => 30,
        ];

        try {
            $connection = new PDO($dsn, $config['username'], $config['password'], $options);

            // 设置连接参数
            $connection->exec("SET SESSION sql_mode = 'STRICT_TRANS_TABLES,NO_ZERO_DATE,NO_ZERO_IN_DATE,ERROR_FOR_DIVISION_BY_ZERO'");
            $connection->exec("SET SESSION innodb_lock_wait_timeout = 5");
            $connection->exec("SET SESSION query_cache_type = ON");

            $connectionId = uniqid($poolName . '_', true);
            $pool['connections'][$connectionId] = $connection;
            $pool['in_use'][$connectionId] = true;
            $pool['created']++;

            return $connection;

        } catch (PDOException $e) {
            throw new RuntimeException("Failed to create database connection: " . $e->getMessage());
        }
    }

    /**
     * 验证连接有效性
     */
    private function validateConnection(PDO $connection): PDO
    {
        try {
            $connection->query("SELECT 1")->fetch();
            return $connection;
        } catch (PDOException $e) {
            throw new RuntimeException("Database connection is invalid: " . $e->getMessage());
        }
    }

    /**
     * 释放连接回连接池
     */
    public function releaseConnection(PDO $connection): void
    {
        foreach ($this->connectionPools as $poolName => &$pool) {
            foreach ($pool['connections'] as $connectionId => $conn) {
                if ($conn === $connection) {
                    unset($pool['in_use'][$connectionId]);
                    $pool['available'][] = $connectionId;

                    // 更新统计
                    if (!isset($this->poolStats[$poolName])) {
                        $this->poolStats[$poolName] = [
                            'total_requests' => 0,
                            'pool_hits' => 0,
                            'new_connections' => 0
                        ];
                    }
                    $this->poolStats[$poolName]['total_requests']++;

                    return;
                }
            }
        }
    }

    /**
     * 连接池健康检查
     */
    public function healthCheck(): array
    {
        $health = [];

        foreach ($this->connectionPools as $poolName => $pool) {
            $healthyConnections = 0;
            $totalConnections = count($pool['connections']);

            foreach ($pool['connections'] as $connection) {
                try {
                    $connection->query("SELECT 1")->fetch();
                    $healthyConnections++;
                } catch (PDOException $e) {
                    // 连接失效，需要重新创建
                    continue;
                }
            }

            $health[$poolName] = [
                'total_connections' => $totalConnections,
                'healthy_connections' => $healthyConnections,
                'available_connections' => count($pool['available']),
                'in_use_connections' => count($pool['in_use']),
                'health_score' => $totalConnections > 0 ? ($healthyConnections / $totalConnections) * 100 : 0
            ];
        }

        return $health;
    }

    /**
     * 清理连接池
     */
    public function cleanup(): void
    {
        foreach ($this->connectionPools as $poolName => &$pool) {
            // 关闭所有连接
            foreach ($pool['connections'] as $connectionId => $connection) {
                try {
                    $connection = null; // 关闭连接
                    $pool['destroyed']++;
                } catch (Exception $e) {
                    // 忽略关闭错误
                }
            }

            $pool['connections'] = [];
            $pool['available'] = [];
            $pool['in_use'] = [];
            $pool['created'] = 0;
        }
    }
}

3. 缓存策略实现

3.1 多层缓存架构

class MultiLevelCacheManager
{
    private array $l1Cache;        // 应用内存缓存
    private Redis $l2Cache;        // Redis分布式缓存
    private PDO $db;               // 数据库连接
    private array $cacheStats = [];

    public function __construct(PDO $db, Redis $redis)
    {
        $this->db = $db;
        $this->l2Cache = $redis;
        $this->l1Cache = [];
    }

    /**
     * 多级缓存获取
     */
    public function get(string $key, callable $dataLoader, array $options = []): mixed
    {
        $startTime = microtime(true);

        // L1缓存：应用内存缓存
        $result = $this->getFromL1Cache($key);
        if ($result !== null) {
            $this->recordCacheHit('l1', microtime(true) - $startTime);
            return $result;
        }

        // L2缓存：Redis分布式缓存
        $result = $this->getFromL2Cache($key);
        if ($result !== null) {
            // 回填L1缓存
            $this->setL1Cache($key, $result, $options['l1_ttl'] ?? 300);
            $this->recordCacheHit('l2', microtime(true) - $startTime);
            return $result;
        }

        // L3缓存：数据库查询缓存
        $result = $this->getFromL3Cache($key, $options);
        if ($result !== null) {
            // 回填L2和L1缓存
            $this->setL2Cache($key, $result, $options['l2_ttl'] ?? 3600);
            $this->setL1Cache($key, $result, $options['l1_ttl'] ?? 300);
            $this->recordCacheHit('l3', microtime(true) - $startTime);
            return $result;
        }

        // 所有缓存都未命中，加载数据
        $result = $dataLoader($this->db);

        // 缓存结果
        $this->setL2Cache($key, $result, $options['l2_ttl'] ?? 3600);
        $this->setL1Cache($key, $result, $options['l1_ttl'] ?? 300);

        // 可选：数据库缓存（用于复杂查询）
        if ($options['l3_cache'] ?? false) {
            $this->setL3Cache($key, $result, $options['l3_ttl'] ?? 7200);
        }

        $this->recordCacheMiss(microtime(true) - $startTime);
        return $result;
    }

    /**
     * L1缓存：应用内存缓存
     */
    private function getFromL1Cache(string $key): mixed
    {
        if (!isset($this->l1Cache[$key])) {
            return null;
        }

        $item = $this->l1Cache[$key];

        // 检查是否过期
        if ($item['expires'] < time()) {
            unset($this->l1Cache[$key]);
            return null;
        }

        return $item['data'];
    }

    private function setL1Cache(string $key, mixed $data, int $ttl): void
    {
        $this->l1Cache[$key] = [
            'data' => $data,
            'expires' => time() + $ttl,
            'created' => time()
        ];

        // LRU：如果缓存过大，移除最旧的条目
        if (count($this->l1Cache) > 1000) {
            $oldestKey = null;
            $oldestTime = time();

            foreach ($this->l1Cache as $cacheKey => $item) {
                if ($item['created'] < $oldestTime) {
                    $oldestTime = $item['created'];
                    $oldestKey = $cacheKey;
                }
            }

            if ($oldestKey) {
                unset($this->l1Cache[$oldestKey]);
            }
        }
    }

    /**
     * L2缓存：Redis分布式缓存
     */
    private function getFromL2Cache(string $key): mixed
    {
        try {
            $data = $this->l2Cache->get("search_cache:{$key}");
            return $data !== false ? unserialize($data) : null;
        } catch (RedisException $e) {
            error_log("Redis cache get failed: " . $e->getMessage());
            return null;
        }
    }

    private function setL2Cache(string $key, mixed $data, int $ttl): void
    {
        try {
            $this->l2Cache->setex("search_cache:{$key}", $ttl, serialize($data));
        } catch (RedisException $e) {
            error_log("Redis cache set failed: " . $e->getMessage());
        }
    }

    /**
     * L3缓存：数据库查询缓存表
     */
    private function getFromL3Cache(string $key, array $options): mixed
    {
        if (!($options['l3_cache'] ?? false)) {
            return null;
        }

        $sql = "SELECT cache_data, expires_at FROM search_query_cache WHERE cache_key = ? AND expires_at > NOW()";
        $stmt = $this->db->prepare($sql);
        $stmt->execute([$key]);

        $result = $stmt->fetch(PDO::FETCH_ASSOC);

        return $result ? unserialize($result['cache_data']) : null;
    }

    private function setL3Cache(string $key, mixed $data, int $ttl): void
    {
        $sql = "INSERT INTO search_query_cache (cache_key, cache_data, expires_at, created_at)
                VALUES (?, ?, DATE_ADD(NOW(), INTERVAL {$ttl} SECOND), NOW())
                ON DUPLICATE KEY UPDATE cache_data = VALUES(cache_data), expires_at = VALUES(expires_at)";

        $stmt = $this->db->prepare($sql);
        $stmt->execute([$key, serialize($data)]);
    }

    /**
     * 缓存预热
     */
    public function warmupCache(array $popularQueries): void
    {
        foreach ($popularQueries as $query) {
            try {
                $key = "search:" . md5($query);
                $this->get($key, function($db) use ($query) {
                    // 执行实际搜索
                    $searchService = new SearchService($db);
                    return $searchService->search($query, ['limit' => 20]);
                }, [
                    'l1_ttl' => 600,    // 10分钟
                    'l2_ttl' => 3600,   // 1小时
                    'l3_ttl' => 7200    // 2小时
                ]);
            } catch (Exception $e) {
                error_log("Cache warmup failed for query: {$query}");
            }
        }
    }

    /**
     * 缓存失效
     */
    public function invalidateCache(array $patterns): void
    {
        // 清除L1缓存
        foreach ($this->l1Cache as $key => $item) {
            foreach ($patterns as $pattern) {
                if (fnmatch($pattern, $key)) {
                    unset($this->l1Cache[$key]);
                    break;
                }
            }
        }

        // 清除L2缓存
        try {
            $redisKeys = [];
            foreach ($patterns as $pattern) {
                $keys = $this->l2Cache->keys("search_cache:" . str_replace('*', '*', $pattern));
                $redisKeys = array_merge($redisKeys, $keys);
            }

            if (!empty($redisKeys)) {
                $this->l2Cache->del($redisKeys);
            }
        } catch (RedisException $e) {
            error_log("Redis cache invalidation failed: " . $e->getMessage());
        }

        // 清除L3缓存
        $sql = "DELETE FROM search_query_cache WHERE cache_key LIKE ?";
        $stmt = $this->db->prepare($sql);

        foreach ($patterns as $pattern) {
            $stmt->execute([str_replace('*', '%', $pattern)]);
        }
    }

    /**
     * 记录缓存统计
     */
    private function recordCacheHit(string $level, float $responseTime): void
    {
        if (!isset($this->cacheStats[$level])) {
            $this->cacheStats[$level] = [
                'hits' => 0,
                'total_time' => 0,
                'avg_time' => 0
            ];
        }

        $this->cacheStats[$level]['hits']++;
        $this->cacheStats[$level]['total_time'] += $responseTime;
        $this->cacheStats[$level]['avg_time'] =
            $this->cacheStats[$level]['total_time'] / $this->cacheStats[$level]['hits'];
    }

    private function recordCacheMiss(float $responseTime): void
    {
        if (!isset($this->cacheStats['misses'])) {
            $this->cacheStats['misses'] = [
                'count' => 0,
                'total_time' => 0,
                'avg_time' => 0
            ];
        }

        $this->cacheStats['misses']['count']++;
        $this->cacheStats['misses']['total_time'] += $responseTime;
        $this->cacheStats['misses']['avg_time'] =
            $this->cacheStats['misses']['total_time'] / $this->cacheStats['misses']['count'];
    }

    /**
     * 获取缓存统计报告
     */
    public function getCacheStats(): array
    {
        $totalRequests = 0;
        $cacheHits = 0;

        foreach ($this->cacheStats as $level => $stats) {
            if ($level !== 'misses') {
                $totalRequests += $stats['hits'];
                $cacheHits += $stats['hits'];
            } else {
                $totalRequests += $stats['count'];
            }
        }

        $hitRate = $totalRequests > 0 ? ($cacheHits / $totalRequests) * 100 : 0;

        return [
            'hit_rate' => round($hitRate, 2),
            'total_requests' => $totalRequests,
            'cache_hits' => $cacheHits,
            'cache_misses' => $this->cacheStats['misses']['count'] ?? 0,
            'stats_by_level' => $this->cacheStats
        ];
    }
}

4. 监控和告警系统

4.1 实时性能监控

class ProductionSearchMonitor
{
    private PDO $db;
    private Redis $redis;
    private array $metrics = [];
    private array $alerts = [];

    public function __construct(PDO $db, Redis $redis)
    {
        $this->db = $db;
        $this->redis = $redis;
    }

    /**
     * 收集系统指标
     */
    public function collectMetrics(): array
    {
        $metrics = [
            'timestamp' => time(),
            'search_performance' => $this->getSearchPerformanceMetrics(),
            'database_performance' => $this->getDatabaseMetrics(),
            'cache_performance' => $this->getCacheMetrics(),
            'system_resources' => $this->getSystemMetrics(),
            'error_rates' => $this->getErrorMetrics()
        ];

        $this->storeMetrics($metrics);
        $this->checkAlerts($metrics);

        return $metrics;
    }

    /**
     * 搜索性能指标
     */
    private function getSearchPerformanceMetrics(): array
    {
        $sql = "SELECT
                    COUNT(*) as total_searches,
                    AVG(execution_time_ms) as avg_response_time,
                    MIN(execution_time_ms) as min_response_time,
                    MAX(execution_time_ms) as max_response_time,
                    AVG(result_count) as avg_result_count,
                    COUNT(*) - SUM(CASE WHEN execution_time_ms <= 100 THEN 1 ELSE 0 END) as slow_searches
                FROM search_metrics
                WHERE timestamp >= DATE_SUB(NOW(), INTERVAL 5 MINUTE)";

        $stmt = $this->db->query($sql);
        $metrics = $stmt->fetch(PDO::FETCH_ASSOC);

        // 计算性能等级
        $metrics['performance_grade'] = $this->calculatePerformanceGrade(
            (float)$metrics['avg_response_time'],
            (int)$metrics['slow_searches'],
            (int)$metrics['total_searches']
        );

        return $metrics;
    }

    /**
     * 数据库性能指标
     */
    private function getDatabaseMetrics(): array
    {
        $metrics = [];

        // 连接数
        $sql = "SHOW STATUS LIKE 'Threads_connected'";
        $result = $this->db->query($sql)->fetch();
        $metrics['active_connections'] = (int)$result['Value'];

        // 慢查询
        $sql = "SHOW STATUS LIKE 'Slow_queries'";
        $result = $this->db->query($sql)->fetch();
        $metrics['slow_queries'] = (int)$result['Value'];

        // 查询缓存命中率
        $sql = "SHOW STATUS LIKE 'Qcache_hits'";
        $result = $this->db->query($sql)->fetch();
        $hits = (int)$result['Value'];

        $sql = "SHOW STATUS LIKE 'Qcache_inserts'";
        $result = $this->db->query($sql)->fetch();
        $inserts = (int)$result['Value'];

        $total = $hits + $inserts;
        $metrics['query_cache_hit_rate'] = $total > 0 ? round(($hits / $total) * 100, 2) : 0;

        // InnoDB缓冲池命中率
        $sql = "SHOW STATUS LIKE 'Innodb_buffer_pool_read_requests'";
        $result = $this->db->query($sql)->fetch();
        $reads = (int)$result['Value'];

        $sql = "SHOW STATUS LIKE 'Innodb_buffer_pool_reads'";
        $result = $this->db->query($sql)->fetch();
        $disk_reads = (int)$result['Value'];

        $total_reads = $reads + $disk_reads;
        $metrics['buffer_pool_hit_rate'] = $total_reads > 0 ? round(($reads / $total_reads) * 100, 2) : 100;

        return $metrics;
    }

    /**
     * 缓存性能指标
     */
    private function getCacheMetrics(): array
    {
        try {
            $info = $this->redis->info();

            return [
                'redis_memory_used' => $info['used_memory_human'],
                'redis_memory_peak' => $info['used_memory_peak_human'],
                'redis_connected_clients' => $info['connected_clients'],
                'redis_keyspace_hits' => $info['keyspace_hits'],
                'redis_keyspace_misses' => $info['keyspace_misses'],
                'redis_hit_rate' => $this->calculateRedisHitRate($info)
            ];
        } catch (RedisException $e) {
            return [
                'redis_status' => 'error',
                'redis_error' => $e->getMessage()
            ];
        }
    }

    /**
     * 系统资源指标
     */
    private function getSystemMetrics(): array
    {
        $metrics = [];

        // CPU使用率
        $load = sys_getloadavg();
        $metrics['cpu_load_1m'] = $load[0] ?? 0;
        $metrics['cpu_load_5m'] = $load[1] ?? 0;
        $metrics['cpu_load_15m'] = $load[2] ?? 0;

        // 内存使用
        $memory = $this->getMemoryUsage();
        $metrics['memory_used_mb'] = $memory['used'];
        $metrics['memory_total_mb'] = $memory['total'];
        $metrics['memory_usage_percent'] = $memory['percent'];

        // 磁盘使用
        $disk = $this->getDiskUsage();
        $metrics['disk_used_gb'] = $disk['used'];
        $metrics['disk_total_gb'] = $disk['total'];
        $metrics['disk_usage_percent'] = $disk['percent'];

        return $metrics;
    }

    /**
     * 错误率指标
     */
    private function getErrorMetrics(): array
    {
        $sql = "SELECT
                    COUNT(*) as total_errors,
                    SUM(CASE WHEN error_message LIKE '%timeout%' THEN 1 ELSE 0 END) as timeout_errors,
                    SUM(CASE WHEN error_message LIKE '%connection%' THEN 1 ELSE 0 END) as connection_errors,
                    SUM(CASE WHEN error_message LIKE '%memory%' THEN 1 ELSE 0 END) as memory_errors
                FROM search_error_logs
                WHERE created_at >= DATE_SUB(NOW(), INTERVAL 5 MINUTE)";

        $result = $this->db->query($sql)->fetch(PDO::FETCH_ASSOC);

        // 计算错误率
        $sql = "SELECT COUNT(*) as total_searches FROM search_metrics WHERE timestamp >= DATE_SUB(NOW(), INTERVAL 5 MINUTE)";
        $totalSearches = (int)$this->db->query($sql)->fetchColumn();

        $result['error_rate'] = $totalSearches > 0 ?
            round(($result['total_errors'] / $totalSearches) * 100, 2) : 0;

        return $result;
    }

    /**
     * 检查告警条件
     */
    private function checkAlerts(array $metrics): void
    {
        $this->alerts = [];

        // 搜索性能告警
        if ($metrics['search_performance']['avg_response_time'] > 500) {
            $this->createAlert('high_response_time', 'warning', [
                'value' => $metrics['search_performance']['avg_response_time'],
                'threshold' => 500
            ]);
        }

        // 数据库连接告警
        if ($metrics['database_performance']['active_connections'] > 800) {
            $this->createAlert('high_db_connections', 'critical', [
                'value' => $metrics['database_performance']['active_connections'],
                'threshold' => 800
            ]);
        }

        // 缓存命中率告警
        if ($metrics['cache_performance']['redis_hit_rate'] < 80) {
            $this->createAlert('low_cache_hit_rate', 'warning', [
                'value' => $metrics['cache_performance']['redis_hit_rate'],
                'threshold' => 80
            ]);
        }

        // 错误率告警
        if ($metrics['error_rates']['error_rate'] > 5) {
            $this->createAlert('high_error_rate', 'critical', [
                'value' => $metrics['error_rates']['error_rate'],
                'threshold' => 5
            ]);
        }

        // 系统资源告警
        if ($metrics['system_resources']['cpu_load_1m'] > 10) {
            $this->createAlert('high_cpu_load', 'warning', [
                'value' => $metrics['system_resources']['cpu_load_1m'],
                'threshold' => 10
            ]);
        }

        if ($metrics['system_resources']['memory_usage_percent'] > 85) {
            $this->createAlert('high_memory_usage', 'critical', [
                'value' => $metrics['system_resources']['memory_usage_percent'],
                'threshold' => 85
            ]);
        }

        // 发送告警
        if (!empty($this->alerts)) {
            $this->sendAlerts();
        }
    }

    /**
     * 创建告警
     */
    private function createAlert(string $type, string $severity, array $data): void
    {
        $this->alerts[] = [
            'type' => $type,
            'severity' => $severity,
            'data' => $data,
            'timestamp' => date('Y-m-d H:i:s'),
            'resolved' => false
        ];
    }

    /**
     * 发送告警通知
     */
    private function sendAlerts(): void
    {
        foreach ($this->alerts as $alert) {
            $message = $this->formatAlertMessage($alert);

            // 记录到数据库
            $this->recordAlert($alert);

            // 发送通知（邮件、Slack等）
            if ($alert['severity'] === 'critical') {
                $this->sendCriticalAlert($message);
            }
        }
    }

    /**
     * 格式化告警消息
     */
    private function formatAlertMessage(array $alert): string
    {
        $typeMessages = [
            'high_response_time' => '搜索响应时间过高: {{value}}ms (阈值: {{threshold}}ms)',
            'high_db_connections' => '数据库连接数过高: {{value}} (阈值: {{threshold}})',
            'low_cache_hit_rate' => '缓存命中率过低: {{value}}% (阈值: {{threshold}}%)',
            'high_error_rate' => '错误率过高: {{value}}% (阈值: {{threshold}}%)',
            'high_cpu_load' => 'CPU负载过高: {{value}} (阈值: {{threshold}})',
            'high_memory_usage' => '内存使用率过高: {{value}}% (阈值: {{threshold}}%)'
        ];

        $template = $typeMessages[$alert['type']] ?? '未知告警类型: {{type}}';
        $message = $template;

        foreach ($alert['data'] as $key => $value) {
            $message = str_replace("{{$key}}", $value, $message);
        }

        return "[{$alert['severity'].toUpperCase()}] {$message} - {$alert['timestamp']}";
    }

    /**
     * 记录告警到数据库
     */
    private function recordAlert(array $alert): void
    {
        $sql = "INSERT INTO system_alerts
                (alert_type, severity, message, alert_data, created_at, resolved)
                VALUES (?, ?, ?, ?, NOW(), 0)";

        $stmt = $this->db->prepare($sql);
        $stmt->execute([
            $alert['type'],
            $alert['severity'],
            $this->formatAlertMessage($alert),
            json_encode($alert['data'])
        ]);
    }

    /**
     * 发送关键告警
     */
    private function sendCriticalAlert(string $message): void
    {
        // 这里可以集成实际的告警系统
        error_log("CRITICAL ALERT: {$message}");

        // 邮件告警
        // mail($adminEmail, '生产环境告警', $message);

        // Slack告警
        // $this->sendSlackAlert($message);
    }

    /**
     * 辅助方法
     */
    private function calculatePerformanceGrade(float $avgTime, int $slowSearches, int $totalSearches): string
    {
        $slowRate = $totalSearches > 0 ? ($slowSearches / $totalSearches) * 100 : 0;

        if ($avgTime <= 100 && $slowRate <= 5) {
            return 'A';  // 优秀
        } elseif ($avgTime <= 200 && $slowRate <= 10) {
            return 'B';  // 良好
        } elseif ($avgTime <= 500 && $slowRate <= 20) {
            return 'C';  // 一般
        } else {
            return 'D';  // 需要优化
        }
    }

    private function calculateRedisHitRate(array $info): float
    {
        $hits = (int)$info['keyspace_hits'];
        $misses = (int)$info['keyspace_misses'];
        $total = $hits + $misses;

        return $total > 0 ? round(($hits / $total) * 100, 2) : 0;
    }

    private function getMemoryUsage(): array
    {
        $memory = memory_get_usage(true);
        $total = 1024 * 1024 * 1024; // 1GB (假设)

        return [
            'used' => round($memory / 1024 / 1024, 2),
            'total' => round($total / 1024 / 1024, 2),
            'percent' => round(($memory / $total) * 100, 2)
        ];
    }

    private function getDiskUsage(): array
    {
        $path = '/'; // 根目录
        $total = disk_total_space($path);
        $free = disk_free_space($path);
        $used = $total - $free;

        return [
            'used' => round($used / 1024 / 1024 / 1024, 2),
            'total' => round($total / 1024 / 1024 / 1024, 2),
            'percent' => round(($used / $total) * 100, 2)
        ];
    }

    private function storeMetrics(array $metrics): void
    {
        $key = "search_metrics:" . date('Y-m-d H:i');
        $this->redis->setex($key, 3600, json_encode($metrics));
    }
}

5. 部署和运维最佳实践

5.1 容器化部署

# Dockerfile - 搜索服务
FROM php:8.1-fpm-alpine

# 安装系统依赖
RUN apk add --no-cache \
    mysql-client \
    redis \
    supervisor \
    nginx

# 安装PHP扩展
RUN docker-php-ext-install \
    pdo_mysql \
    mysqli \
    opcache \
    bcmath \
    json

# 安装Redis扩展
RUN pecl install redis && docker-php-ext-enable redis

# 配置PHP
COPY php.ini /usr/local/etc/php/conf.d/custom.ini

# 复制应用代码
COPY . /var/www/html

# 设置权限
RUN chown -R www-data:www-data /var/www/html

# 配置Supervisor
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf

# 配置Nginx
COPY nginx.conf /etc/nginx/nginx.conf

# 暴露端口
EXPOSE 80

# 启动服务
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]

# docker-compose.yml
version: '3.8'

services:
  search-web:
    build: .
    ports:
      - "80:80"
    environment:
      - DB_HOST=mysql-master
      - DB_NAME=search_engine
      - DB_USER=search_user
      - DB_PASS=${DB_PASSWORD}
      - REDIS_HOST=redis
    depends_on:
      - mysql-master
      - redis
    volumes:
      - ./logs:/var/log
    restart: unless-stopped

  mysql-master:
    image: mysql:8.0
    environment:
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
      - MYSQL_DATABASE=search_engine
      - MYSQL_USER=search_user
      - MYSQL_PASSWORD=${DB_PASSWORD}
    volumes:
      - mysql_master_data:/var/lib/mysql
      - ./mysql/master.cnf:/etc/mysql/conf.d/master.cnf
      - ./sql/schema.sql:/docker-entrypoint-initdb.d/schema.sql
    command: --server-id=1 --log-bin=mysql-bin --binlog-format=ROW
    restart: unless-stopped

  mysql-slave:
    image: mysql:8.0
    environment:
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
    volumes:
      - mysql_slave_data:/var/lib/mysql
      - ./mysql/slave.cnf:/etc/mysql/conf.d/slave.cnf
    command: --server-id=2 --relay-log=mysql-relay --read-only=1
    depends_on:
      - mysql-master
    restart: unless-stopped

  redis:
    image: redis:7-alpine
    command: redis-server --appendonly yes --maxmemory 512mb --maxmemory-policy allkeys-lru
    volumes:
      - redis_data:/data
    restart: unless-stopped

  prometheus:
    image: prom/prometheus
    ports:
      - "9090:9090"
    volumes:
      - ./monitoring/prometheus.yml:/etc/prometheus/prometheus.yml
    restart: unless-stopped

  grafana:
    image: grafana/grafana
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
    volumes:
      - grafana_data:/var/lib/grafana
    restart: unless-stopped

volumes:
  mysql_master_data:
  mysql_slave_data:
  redis_data:
  grafana_data:

5.2 自动化部署脚本

#!/bin/bash
# deploy.sh - 生产环境部署脚本

set -e

# 配置变量
ENVIRONMENT=${1:-production}
BACKUP_DIR="/backups/$(date +%Y%m%d_%H%M%S)"
DEPLOY_BRANCH="main"
HEALTH_CHECK_URL="http://localhost/api/health"
SLACK_WEBHOOK_URL="${SLACK_WEBHOOK_URL}"

# 颜色输出
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

log_info() {
    echo -e "${GREEN}[INFO]${NC} $1"
}

log_warn() {
    echo -e "${YELLOW}[WARN]${NC} $1"
}

log_error() {
    echo -e "${RED}[ERROR]${NC} $1"
}

# 发送Slack通知
send_slack_notification() {
    local message=$1
    local color=$2

    curl -X POST -H 'Content-type: application/json' \
        --data "{\"text\":\"$message\", \"color\":\"$color\"}" \
        "$SLACK_WEBHOOK_URL" || log_error "Failed to send Slack notification"
}

# 健康检查
health_check() {
    local max_attempts=30
    local attempt=1

    while [ $attempt -le $max_attempts ]; do
        if curl -f -s "$HEALTH_CHECK_URL" > /dev/null; then
            log_info "Health check passed on attempt $attempt"
            return 0
        fi

        log_warn "Health check failed, attempt $attempt/$max_attempts"
        sleep 10
        ((attempt++))
    done

    log_error "Health check failed after $max_attempts attempts"
    return 1
}

# 数据库备份
backup_database() {
    log_info "Creating database backup..."

    mkdir -p "$BACKUP_DIR"

    # 备份主数据库
    mysqldump -h mysql-master -u root -p"$MYSQL_ROOT_PASSWORD" \
        --single-transaction --routines --triggers \
        search_engine > "$BACKUP_DIR/master_backup.sql"

    # 备份搜索索引
    mysqldump -h mysql-master -u root -p"$MYSQL_ROOT_PASSWORD" \
        --single-transaction \
        search_engine index_tokens index_entries > "$BACKUP_DIR/index_backup.sql"

    log_info "Database backup completed: $BACKUP_DIR"
}

# 部署应用
deploy_application() {
    log_info "Starting application deployment..."

    # 拉取最新代码
    git pull origin "$DEPLOY_BRANCH"

    # 安装依赖
    composer install --no-dev --optimize-autoloader

    # 运行数据库迁移
    php migrations/migrate.php

    # 构建和重启容器
    docker-compose down
    docker-compose build --no-cache
    docker-compose up -d

    log_info "Application deployment completed"
}

# 验证部署
verify_deployment() {
    log_info "Verifying deployment..."

    # 等待服务启动
    sleep 30

    # 健康检查
    if ! health_check; then
        log_error "Deployment verification failed"
        rollback_deployment
        send_slack_notification "❌ Deployment failed and rolled back" "danger"
        exit 1
    fi

    # 运行集成测试
    if ! php tests/integration/run.php; then
        log_warn "Integration tests failed, but deployment continues"
    fi

    log_info "Deployment verification passed"
}

# 回滚部署
rollback_deployment() {
    log_info "Starting deployment rollback..."

    # 恢复数据库备份
    mysql -h mysql-master -u root -p"$MYSQL_ROOT_PASSWORD" \
        search_engine < "$BACKUP_DIR/master_backup.sql"

    # 重启服务（使用上一个镜像）
    docker-compose down
    docker-compose up -d

    log_info "Rollback completed"
}

# 性能测试
performance_test() {
    log_info "Running performance tests..."

    # 运行搜索性能测试
    php tests/performance/search_test.php --iterations=100 --concurrency=10

    # 检查响应时间
    avg_time=$(php tests/performance/response_time_test.php)

    if (( $(echo "$avg_time > 500" | bc -l) )); then
        log_warn "High response time detected: ${avg_time}ms"
    else
        log_info "Performance test passed: ${avg_time}ms average"
    fi
}

# 清理旧的备份
cleanup_old_backups() {
    log_info "Cleaning up old backups..."

    find /backups -type d -mtime +7 -exec rm -rf {} + || true

    log_info "Backup cleanup completed"
}

# 主部署流程
main() {
    log_info "Starting deployment to $ENVIRONMENT environment..."
    send_slack_notification "🚀 Starting deployment to $ENVIRONMENT" "good"

    # 检查环境
    if [ "$ENVIRONMENT" != "production" ] && [ "$ENVIRONMENT" != "staging" ]; then
        log_error "Invalid environment: $ENVIRONMENT"
        exit 1
    fi

    # 创建备份
    backup_database

    # 部署应用
    deploy_application

    # 验证部署
    verify_deployment

    # 性能测试
    performance_test

    # 清理备份
    cleanup_old_backups

    log_info "Deployment completed successfully!"
    send_slack_notification "✅ Deployment to $ENVIRONMENT completed successfully" "good"
}

# 错误处理
trap 'log_error "Deployment failed!"; send_slack_notification "❌ Deployment failed" "danger"; exit 1' ERR

# 执行主流程
main

6. 本章总结

6.1 核心收获

部署架构：
– 掌握主从复制和读写分离的设计
– 学会负载均衡和高可用架构
– 理解容器化部署的最佳实践
– 掌握自动化部署流程设计

性能调优：
– 掌握MySQL生产环境参数优化
– 学会数据库连接池的设计和管理
– 理解多层缓存架构的实现
– 掌握查询性能监控和优化技巧

监控运维：
– 建立完善的系统监控体系
– 实现智能告警和故障检测
– 掌握自动化运维流程设计
– 学会容量规划和扩展策略

生产实践：
– 构建完整的CI/CD部署流程
– 实现零停机的滚动更新
– 建立灾备和恢复机制
– 掌握安全运维的最佳实践

6.2 系列教程总结

经过6章的深入学习，你已经掌握了：

第1章：理解了简洁搜索系统的设计哲学
第2章：掌握了Tokenization的核心技术
第3章：学会了科学的权重系统设计
第4章：构建了高性能的索引架构
第5章：实现了优化的查询和排序系统
第6章：建立了完整的生产环境部署方案

6.3 下一步发展方向

技术深度：
– 集成机器学习算法提升搜索准确性
– 实现实时搜索和流式索引更新
– 探索分布式搜索和大数据处理
– 研究语义搜索和自然语言处理

架构演进：
– 微服务化搜索架构设计
– 多云部署和混合云策略
– 边缘计算和CDN集成
– 容器编排和Kubernetes部署

业务应用：
– 个性化推荐系统
– 智能搜索建议
– 语音搜索集成
– 多语言搜索支持

结语：搜索引擎的设计和实现是一个持续优化的过程。本系列为你提供了坚实的基础，但真正的成长来自于实践中的不断探索和改进。保持好奇心，继续学习，你将能够构建出更强大、更智能的搜索系统。

上一篇 → 第05章：搜索查询优化 | 系列完成 → 返回系列首页