Quantcast
Channel: Active questions tagged ubuntu - Stack Overflow
Viewing all articles
Browse latest Browse all 5956

Problem with Postgresql efficiency on ubuntu server

$
0
0

I have two identical DB on my virtual server and my pc.I made dump db on server and then unpack it on my home pcI noticed that the same query is executed in different times on my pc and serveron pc ~ 0.9 sec, on vps ~20 sec

Configeration of PC:Core(TM) i7-13700K   3.40 GHz128gb RAM2tb ssd m2
Configeration of vps:CPUs 4 10.4 GHz allocated64gb RAM2tb hdd

I understand that my disk is much faster than on vps and even if I install new ssd on vps I won't achieve 0.9 sec, but still it should be fast than 20 sec.

So I install 250gb ssd on vps, yhen move postgres cluster on it and nothing happend, same 20secthen I changed postgresql.confI set:

shared_buffers = 16GBhuge_pages = onwork_mem = 300MBmaintenance_work_mem = 8GBeffective_io_concurrency = 200random_page_cost = 1.0effective_cache_size = 32GB

Also I changed kernel settingsI set

vm.nr_hugepages bases on system needskernel.shmmax=16777216kernel.shmall=16777216vm.swappiness=1vm.overcommit_memory=2

Also I ran pg_stat_reset() and ANALYZE on vps dbAnd interesting fact, that after pg_stat_reset() this query was executed in 2 sec, but then for the second and next times again 20 sec.

It means it is possible to execute query faster, but I don't understand how

query example:

selecty.node_id,y.linksfrom (select distinctcase when x.point_num = 1 then x.ref_node_id when x.point_num = x.max_p then x.non_ref_node_id end as node_id,array_agg(distinct x.link_id) as links,array_agg(distinct x.geom) as geomsfrom (select lk.link_id,     (ST_DumpPoints(lk.geom)).path[1] as point_num,     ST_NPoints(lk.geom) as max_p,     lk.ref_node_id,     lk.non_ref_node_id,     (ST_DumpPoints(lk.geom)).geom as geom    from "Link" lk      ) as xwhere x.point_num = 1 or x.point_num = x.max_pgroup by node_id) as ywhere array_length(y.geoms, 1) > 1

explain (analyze, buffers, settings) on PC

"Subquery Scan on y  (cost=195715818275.45..195715819491.81 rows=60818 width=40) (actual time=53749.993..53750.033 rows=0 loops=1)""  Buffers: shared hit=11625 read=594820, temp read=754031 written=755068""  ->  Unique  (cost=195715818275.45..195715818883.63 rows=60818 width=72) (actual time=53749.992..53750.032 rows=0 loops=1)""        Buffers: shared hit=11625 read=594820, temp read=754031 written=755068""        ->  Sort  (cost=195715818275.45..195715818427.50 rows=60818 width=72) (actual time=53749.991..53750.031 rows=0 loops=1)""              Sort Key: (CASE WHEN (x.point_num = 1) THEN x.ref_node_id WHEN (x.point_num = x.max_p) THEN x.non_ref_node_id ELSE NULL::bigint END), (array_agg(DISTINCT x.link_id)), (array_agg(DISTINCT x.geom))""              Sort Method: quicksort  Memory: 25kB""              Buffers: shared hit=11625 read=594820, temp read=754031 written=755068""              ->  GroupAggregate  (cost=195715805131.36..195715810947.29 rows=60818 width=72) (actual time=53749.985..53750.025 rows=0 loops=1)""                    Group Key: (CASE WHEN (x.point_num = 1) THEN x.ref_node_id WHEN (x.point_num = x.max_p) THEN x.non_ref_node_id ELSE NULL::bigint END)""                    Filter: (array_length(array_agg(DISTINCT x.geom), 1) > 1)""                    Rows Removed by Filter: 15073003""                    Buffers: shared hit=11625 read=594820, temp read=754031 written=755068""                    ->  Sort  (cost=195715805131.36..195715805597.06 rows=186278 width=48) (actual time=30583.674..32904.574 rows=37398936 loops=1)""                          Sort Key: (CASE WHEN (x.point_num = 1) THEN x.ref_node_id WHEN (x.point_num = x.max_p) THEN x.non_ref_node_id ELSE NULL::bigint END)""                          Sort Method: external merge  Disk: 2010792kB""                          Buffers: shared hit=11625 read=594820, temp read=754031 written=755068""                          ->  Subquery Scan on x  (cost=1000.00..195715783092.43 rows=186278 width=48) (actual time=0.412..19751.531 rows=37398936 loops=1)""                                Filter: ((x.point_num = 1) OR (x.point_num = x.max_p))""                                Rows Removed by Filter: 73240944""                                Buffers: shared hit=11625 read=594820""                                ->  Gather  (cost=1000.00..195715502043.18 rows=18674524 width=64) (actual time=0.411..14311.958 rows=110639880 loops=1)""                                      Workers Planned: 2""                                      Workers Launched: 2""                                      Buffers: shared hit=11625 read=594820""                                      ->  Result  (cost=0.00..195713633590.78 rows=7781052000 width=64) (actual time=0.296..13555.925 rows=36879960 loops=3)""                                            Buffers: shared hit=11625 read=594820""                                            ->  ProjectSet  (cost=0.00..136891570.78 rows=7781052000 width=177) (actual time=0.291..8554.085 rows=36879960 loops=3)""                                                  Buffers: shared hit=11625 read=594820""                                                  ->  Parallel Seq Scan on ""Link"" lk  (cost=0.00..684255.52 rows=7781052 width=145) (actual time=0.276..843.207 rows=6233156 loops=3)""                                                        Buffers: shared hit=11625 read=594820""Settings: search_path = '""$user"", public, tiger, topology'""Planning Time: 0.230 ms""Execution Time: 53838.300 ms"

explain (analyze, buffers, settings) on vps

"Subquery Scan on y  (cost=390643534813.38..390643536030.92 rows=60877 width=40) (actual time=302571.382..302584.406 rows=35 loops=1)""  Buffers: shared hit=36 read=615858, temp read=251588 written=251589""  ->  Unique  (cost=390643534813.38..390643535422.15 rows=60877 width=72) (actual time=302571.363..302584.367 rows=35 loops=1)""        Buffers: shared hit=36 read=615858, temp read=251588 written=251589""        ->  Sort  (cost=390643534813.38..390643534965.57 rows=60877 width=72) (actual time=302571.345..302575.100 rows=35 loops=1)""              Sort Key: (CASE WHEN (x.point_num = 1) THEN x.ref_node_id WHEN (x.point_num = x.max_p) THEN x.non_ref_node_id ELSE NULL::bigint END), (array_agg(DISTINCT x.link_id)), (array_agg(DISTINCT x.geom))""              Sort Method: quicksort  Memory: 31kB""              Buffers: shared hit=36 read=615858, temp read=251588 written=251589""              ->  GroupAggregate  (cost=390643524154.35..390643529975.60 rows=60877 width=72) (actual time=208002.806..302574.125 rows=35 loops=1)""                    Group Key: (CASE WHEN (x.point_num = 1) THEN x.ref_node_id WHEN (x.point_num = x.max_p) THEN x.non_ref_node_id ELSE NULL::bigint END)""                    Filter: (array_length(array_agg(DISTINCT x.geom), 1) > 1)""                    Rows Removed by Filter: 15073476""                    Buffers: shared hit=28 read=615858, temp read=251588 written=251589""                    ->  Sort  (cost=390643524154.35..390643524620.41 rows=186424 width=48) (actual time=207912.909..215920.138 rows=37399512 loops=1)""                          Sort Key: (CASE WHEN (x.point_num = 1) THEN x.ref_node_id WHEN (x.point_num = x.max_p) THEN x.non_ref_node_id ELSE NULL::bigint END)""                          Sort Method: external merge  Disk: 2012704kB""                          Buffers: shared hit=19 read=615858, temp read=251588 written=251589""                          ->  Subquery Scan on x  (cost=1000.00..390643507834.58 rows=186424 width=48) (actual time=1159.748..161684.475 rows=37399512 loops=1)""                                Filter: ((x.point_num = 1) OR (x.point_num = x.max_p))""                                Rows Removed by Filter: 73242457""                                Buffers: shared hit=19 read=615858""                                ->  Gather  (cost=1000.00..390643226566.32 rows=18689076 width=64) (actual time=1159.744..139305.430 rows=110641969 loops=1)""                                      Workers Planned: 2""                                      Workers Launched: 2""                                      Buffers: shared hit=19 read=615858""                                      ->  Result  (cost=0.00..390641356658.72 rows=7787115000 width=64) (actual time=1090.274..90631.003 rows=36880656 loops=3)""                                            Buffers: shared hit=19 read=615858""                                            ->  ProjectSet  (cost=0.00..234346133.73 rows=7787115000 width=176) (actual time=1090.264..61234.001 rows=36880656 loops=3)""                                                  Buffers: shared hit=19 read=615858""                                                  ->  Parallel Seq Scan on ""Link"" lk  (cost=0.00..693748.15 rows=7787115 width=144) (actual time=6.453..10329.767 rows=6233252 loops=3)""                                                        Buffers: shared hit=19 read=615858""Settings: effective_cache_size = '32GB', effective_io_concurrency = '500', random_page_cost = '1', work_mem = '2GB'""Planning:""  Buffers: shared hit=188""Planning Time: 0.938 ms""JIT:""  Functions: 44""  Options: Inlining true, Optimization true, Expressions true, Deforming true""  Timing: Generation 9.531 ms, Inlining 524.496 ms, Optimization 1881.259 ms, Emission 843.753 ms, Total 3259.039 ms""Execution Time: 304232.738 ms"

Viewing all articles
Browse latest Browse all 5956

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>