有句 SQL 性能很差,不知道怎么优化,求助下大家? - V2EX
V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
HelloAmadeus
V2EX    数据库

有句 SQL 性能很差,不知道怎么优化,求助下大家?

  •  
  •   HelloAmadeus 2024-11-04 20:01:21 +08:00 2478 次点击
    这是一个创建于 414 天前的主题,其中的信息可能已经有所发展或是发生改变。
    select DATE_FORMAT(`date`, "%Y-%m-%d") as time, thealth_level as "健康度", COUNT(DISTINCT(tserver_name)) as "总数" from ( select date, server_name as tserver_name, MIN(health_level) as thealth_level FROM cpu_throttled_health WHERE date > DATE_FORMAT(CURRENT_DATE() - INTERVAL 7 DAY, "%Y-%m-%d") GROUP BY date, server_name ) as tt group by tt.thealth_level, tt.date order by date,thealth_level 

    已经给 date,sever_name 增加了 index ,explain 显示能用到 index ,但是速度还是很慢,不知道有什么可以优化的。

    数据是每分钟统计服务一个 health_level 指标,目的是按天统计服务数,计算每个 health_level 下有几个服务,一个服务按当天最小 health_level 计算分组。

    目前能想到的点就是不要实时计算,每天离线统计昨天的数据,存在另外一个表里。不知道有没有可以优化的地方,能实时读。

    8 条回复    2024-11-05 12:16:54 +08:00
    HowardTang
        1
    HowardTang  
       2024-11-04 20:04:19 +08:00
    来自 AI 的建议:
    我来帮你分析和优化这个查询。以下是几个优化建议:
    1. 首先可以把子查询提取出来作为一个临时表,减少重复计算:

    WITH daily_min_health AS (
    SELECT
    DATE_FORMAT(`date`, "%Y-%m-%d") as date,
    server_name,
    MIN(health_level) as health_level
    FROM cpu_throttled_health
    WHERE date > DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY)
    GROUP BY DATE_FORMAT(`date`, "%Y-%m-%d"), server_name
    )
    SELECT
    date,
    health_level as "健康度",
    COUNT(DISTINCT server_name) as "总数"
    FROM daily_min_health
    GROUP BY date, health_level
    ORDER BY date, health_level;


    2.可以创建复合索引来优化:
    CREATE INDEX idx_date_server_health ON cpu_throttled_health(date, server_name, health_level);

    3.如果需要实时查询,可以考虑增加物化视图:
    CREATE MATERIALIZED VIEW mv_daily_health_stats
    REFRESH ON COMMIT AS
    SELECT
    DATE_FORMAT(`date`, "%Y-%m-%d") as date,
    health_level,
    COUNT(DISTINCT server_name) as server_count
    FROM daily_min_health
    GROUP BY DATE_FORMAT(`date`, "%Y-%m-%d"), health_level;

    4.分区表优化:
    ALTER TABLE cpu_throttled_health
    PARTITION BY RANGE (TO_DAYS(date)) (
    PARTITION p_history VALUES LESS THAN (TO_DAYS('2024-01-01')),
    PARTITION p_current VALUES LESS THAN MAXVALUE
    );

    主要优化思路:
    使用复合索引提升查询效率
    考虑物化视图或预计算
    使用分区表减少扫描范围
    如果可以接受,建议改为每天离线计算更新统计表
    Jinnrry
        2
    Jinnrry  
       2024-11-04 20:05:14 +08:00
    1 、什么数据库
    2 、表结构贴上来
    3 、数据量多少啊

    这 3 个都不知道,盲猜子查询,group by 性能问题。你先试试子查询耗时,然后再加上 group by 看耗时
    yeqizhang
        3
    yeqizhang  
       2024-11-04 20:05:51 +08:00 via Android
    数据量? tt 那个查询查询多慢?
    irisdev
        4
    irisdev  
       2024-11-04 20:07:28 +08:00
    这个子查询意义何在
    ntedshen
        5
    ntedshen  
       2024-11-04 21:28:42 +08:00
    如果没记错那么 health_level 同样需要索引。。。
    以及 date 直接存时间戳然后-86400*7 得了,一堆日期函数感觉是花拳绣腿。。。
    ryan961
        6
    ryan961  
       2024-11-05 09:25:35 +08:00
    看看这个 https://mp.weixin.qq.com/s/Gr3yk7J1XSe6QCmPmvIjWg ,当中提到"用双重 group by 代替 count(distinct)" 以及一些其他优化方式,具体我没试过,但感觉可能对你有点用。
    EthanZC
        7
    EthanZC  
       2024-11-05 12:06:09 +08:00
    明明可以不用子查询,为啥非得套一层,还有这个 group by tt.date, select 又是 DATE_FORMAT(`date`, "%Y-%m-%d"),这种语法也就 MySQL 能让你这么干,其实非常不建议.若是实在要用,在子查询里,就 DATE_FORMAT(`date`, "%Y-%m-%d") 转换好格式再在外面的查询上面直接用
    zizon
        8
    zizon  
       2024-11-05 12:16:54 +08:00
    distinct 可以不用吧?子查询已经保证了明天每个 server 只有一条 health record.
    关于     帮助文档     自助推广系统     博客     API     FAQ     Solana     3188 人在线   最高记录 6679       Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 37ms UTC 10:57 PVG 18:57 LAX 02:57 JFK 05:57
    Do have faith in what you're doing.
    ubao msn snddm index pchome yahoo rakuten mypaper meadowduck bidyahoo youbao zxmzxm asda bnvcg cvbfg dfscv mmhjk xxddc yybgb zznbn ccubao uaitu acv GXCV ET GDG YH FG BCVB FJFH CBRE CBC GDG ET54 WRWR RWER WREW WRWER RWER SDG EW SF DSFSF fbbs ubao fhd dfg ewr dg df ewwr ewwr et ruyut utut dfg fgd gdfgt etg dfgt dfgd ert4 gd fgg wr 235 wer3 we vsdf sdf gdf ert xcv sdf rwer hfd dfg cvb rwf afb dfh jgh bmn lgh rty gfds cxv xcv xcs vdas fdf fgd cv sdf tert sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf shasha9178 shasha9178 shasha9178 shasha9178 shasha9178 liflif2 liflif2 liflif2 liflif2 liflif2 liblib3 liblib3 liblib3 liblib3 liblib3 zhazha444 zhazha444 zhazha444 zhazha444 zhazha444 dende5 dende denden denden2 denden21 fenfen9 fenf619 fen619 fenfe9 fe619 sdf sdf sdf sdf sdf zhazh90 zhazh0 zhaa50 zha90 zh590 zho zhoz zhozh zhozho zhozho2 lislis lls95 lili95 lils5 liss9 sdf0ty987 sdft876 sdft9876 sdf09876 sd0t9876 sdf0ty98 sdf0976 sdf0ty986 sdf0ty96 sdf0t76 sdf0876 df0ty98 sf0t876 sd0ty76 sdy76 sdf76 sdf0t76 sdf0ty9 sdf0ty98 sdf0ty987 sdf0ty98 sdf6676 sdf876 sd876 sd876 sdf6 sdf6 sdf9876 sdf0t sdf06 sdf0ty9776 sdf0ty9776 sdf0ty76 sdf8876 sdf0t sd6 sdf06 s688876 sd688 sdf86