Tumbleweed Running ~20% Slower vs other OSes

Pheven42069 · November 8, 2024, 4:53am

Howdy,

I’ve got a handful of Dell Latitude 5520s that I’ve been tasked with chucking Linux on and Tumbleweed seemed like a good idea to try out. Everything installs fine but I noticed having a couple of them running Ubuntu that the Tumbleweed machines were consistently ~20% slower running stuff like Blender.

The good ol’ BMW render takes consistently around 2 minutes and 30 seconds on Ubuntu 24.04.1 LTS and 24.10, as well as Fedora Workstation 41. The same machines when Tumbleweed is installed complete the same render in about 3 minutes and 15 seconds. I didn’t have a Leap ISO handy but its downloading now so I can give it a crack.

I had a look at clock speeds and thermals to see if it was a throttling thing, there doesn’t appear to be any thermal throttling going on with any of the OSes. All of them jump into PL2 when an all core load starts and run at 4GHz until PL1 kicks in and they drop down to the base clock of 2.6GHz. They sometimes go up a little bit, I’ve seen 2.9GHz all-core while running the s-tui stress test while staying within the ~28W PL1 limit.

Its not the same in Tumbleweed though, clock speeds are consistently 400MHz lower under an all-core load. 2.2GHz during Blender and 2.5GHz under the s-tui stress test. The clock speed difference makes the performance difference check out.

I’ve tried a couple of things, the easiest being the power profile presented in Gnome. The SMBIOS power profiles don’t make a difference, neither does changing the cpufreq governor between performance and powersave. Disabling intel_pstate in GRUB and using acpi-cpufreq also hasn’t done the trick. If anything it seems to flip between 2.1GHz and 2.2GHz more without intel_pstate.

Any ideas? I like Tumbleweed so far and would like to keep the new software thing.

marel · November 8, 2024, 7:12am

Welcome to the forum good you found out it has to do with processor speed being limited in Tumbleweed.

Looks like you are running on a laptop with a ~28W limit. I am not sure how you detected the cpu clocks are lower (which tools) but can you run something I am familiar with and that has nice output:

sudo turbostat --Summary --quiet --show Busy%,Avg_MHz,PkgTmp,PkgWatt --interval 10

Start it before your test and stop it when the test is done using Ctrl-C.
Would be great if you also could do it for Ubuntu/Fedora.

You can use “sudo cpupower frequency-info” to check which governors you are using.

Pheven42069 · November 9, 2024, 12:31am

Hi marel,

Sure can, here’s what turbostat reckons when running a Blender job on Tumbleweed. The clock speeds match what I’m seeing in btop, s-tui, the Vitals Gnome extension, etc.

localhost:~> sudo turbostat --Summary --quiet --show Busy%,Avg_MHz,PkgTmp,PkgWatt --interval 10
Avg_MHz	Busy%	PkgTmp	PkgWatt
25	2.26	40	2.07
30	2.61	40	2.12
25	2.28	40	2.03
25	2.23	40	1.99
91	4.95	43	3.20
3396	85.02	83	39.61
2208	99.75	60	14.92
2146	99.75	62	13.74
2179	99.74	64	13.89
2174	99.75	65	13.94
2165	99.74	65	13.94
2168	99.75	67	13.95
2141	99.71	67	13.95
2149	99.74	68	13.95
2157	99.75	69	13.95
2170	99.75	69	13.95
2166	99.76	67	13.95
2163	99.74	67	13.95
2178	99.74	66	13.95
2180	99.74	65	13.95
2184	99.75	66	13.95
2183	99.75	65	13.94
2185	99.74	66	13.95
2178	99.75	65	13.95
2186	99.75	65	13.95
573	26.41	57	5.23
25	1.71	53	2.19
33	2.14	52	2.40

I set the governor back to the default powersave option after testing if setting it to performance had any impact. Confirmed that it doesn’t.

localhost:~> sudo cpupower frequency-info
analyzing CPU 5:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 5
  CPUs which need to have their frequency coordinated by software: 5
  maximum transition latency:  Cannot determine or is not supported.
  hardware limits: 400 MHz - 4.40 GHz
  available cpufreq governors: performance powersave
  current policy: frequency should be within 400 MHz and 4.40 GHz.
                  The governor "powersave" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 1.09 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: yes

I’ll fire up Fedora and do the same there. Will provide an edit in a few minutes.

Cheers for having a look, pretty interest to see what the go is.

Pheven42069 · November 9, 2024, 12:56am

There’s no option to edit a post apparently, sorry for the double reply.

Here’s the turbostat results for Blender under Fedora and the output of cpupower.

fedora:~$ sudo turbostat --Summary --quiet --show Busy%,Avg_MHz,PkgTmp,PkgWatt --interval 10
Avg_MHz	Busy%	PkgTmp	PkgWatt
48	3.12	45	2.56
29	2.26	44	2.09
182	6.40	46	4.74
3448	86.47	89	39.35
2550	99.75	67	19.31
2312	99.75	69	14.75
2850	99.75	83	22.23
3062	99.75	75	25.18
2533	99.76	75	17.41
2539	99.76	74	17.44
2539	99.72	74	17.44
2542	99.75	74	17.45
2543	99.76	74	17.45
2546	99.76	73	17.44
2549	99.75	73	17.45
2551	99.71	72	17.45
2555	99.75	73	17.45
2556	99.72	73	17.45
2557	99.70	72	17.45
890	35.20	60	7.51
26	1.91	56	2.31
31	2.16	59	2.32


fedora:~$ sudo cpupower frequency-info 
analyzing CPU 2:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 2
  CPUs which need to have their frequency coordinated by software: 2
  maximum transition latency:  Cannot determine or is not supported.
  hardware limits: 400 MHz - 4.40 GHz
  available cpufreq governors: performance powersave
  current policy: frequency should be within 400 MHz and 4.40 GHz.
                  The governor "powersave" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 2.66 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: yes

arvidjaar · November 9, 2024, 5:30am

What is the output of

cat /sys/devices/system/cpu/intel_pstate/energy_performance_preference
cat /sys/devices/system/cpu/intel_pstate/energy_performance_available_preferences

in both cases?

marel · November 9, 2024, 7:03am

Thanks for posting the output of turbostat for the two OS’s:

Comparing the entry with the largest temperature:

Avg_MHz	Busy%	PkgTmp	PkgWatt   OS
2170	99.75	69	    13.95     Tumbleweed
2850	99.75	83	    22.23     Fedora

I tried the command suggested by @arvidjaar but that file is not on my system, there are other files:

> cd /sys/devices/system/cpu/intel_pstate/
> grep . *
energy_efficiency:0
hwp_dynamic_boost:0
max_perf_pct:100
min_perf_pct:16
no_turbo:0
num_pstates:42
status:active
turbo_pct:32

This is when my desktop running an i9700 is running “idle”.

Not my choice, the edit time-out is 10 minutes

arvidjaar · November 9, 2024, 7:51am

It depends on the CPU support.

Pheven42069 · November 9, 2024, 8:48am

arvidjaar:

What is the output of

cat /sys/devices/system/cpu/intel_pstate/energy_performance_preference
cat /sys/devices/system/cpu/intel_pstate/energy_performance_available_preferences

in both cases?

I don’t have those files on these 1145G7s either unfortunately.

marel:

Thanks for posting the output of turbostat for the two OS’s:

Comparing the entry with the largest temperature:
Avg_MHz	Busy%	PkgTmp	PkgWatt   OS
2170	99.75	69	    13.95     Tumbleweed
2850	99.75	83	    22.23     Fedora
I tried the command suggested by @arvidjaar but that file is not on my system, there are other files:
> cd /sys/devices/system/cpu/intel_pstate/
> grep . *
energy_efficiency:0
hwp_dynamic_boost:0
max_perf_pct:100
min_perf_pct:16
no_turbo:0
num_pstates:42
status:active
turbo_pct:32
This is when my desktop running an i9700 is running “idle”.

Pheven42069:

There’s no option to edit a post apparently, sorry for the double reply.

Not my choice, the edit time-out is 10 minutes

Not sure that the ten second interval was sufficient to capture the PL2 boost accurately. Seems like Tau is only ~8 seconds. Here’s something a little more granular, cut just after the CPU settles into PL1 boost.

localhost:~> sudo turbostat --Summary --quiet --show Busy%,Avg_MHz,PkgTmp,PkgWatt --interval 0.5
Avg_MHz	Busy%	PkgTmp	PkgWatt
206	7.59	47	4.59
84	6.19	47	3.61
91	6.75	47	3.31
44	3.73	47	2.44
58	4.43	47	2.84
214	7.71	47	5.90
47	3.81	47	2.53
146	5.44	61	3.53
2276	56.91	76	28.51
3975	99.37	81	44.50
3991	99.77	83	44.56
3989	99.72	83	45.54
3980	99.77	84	45.21
3999	99.71	83	45.73
3991	99.77	86	45.51
3990	99.74	86	45.62
3991	99.77	86	45.59
3991	99.77	87	46.18
3989	99.73	86	46.05
3991	99.77	87	45.86
3984	99.61	87	45.48
3991	99.77	89	46.29
3990	99.75	89	46.73
3991	99.77	89	46.81
3990	99.75	91	46.80
3621	99.77	72	38.30
2282	99.77	66	14.71
2095	99.74	65	13.09
2095	99.77	66	13.07
2087	99.77	65	13.35
2035	99.77	65	12.91
2083	99.77	65	13.33
2049	99.77	65	13.04
2060	99.77	66	13.40

s-tui makes it a little prettier. You can see the spike up to 60W while PL2 is active before temperature and power limits start arm wrestling the clock frequency into the PL1 state.

Interestingly the short duration one/two core boost clocks still hit 4.4GHz in Tumbleweed. Had a look at what the power limits and tau values are supposed to be but they don’t make much sense.

localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/name
package-0
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_0_name 
long_term
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_0_power_limit_uw 
60000000
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_0_max_power_uw 
28000000
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_0_time_window_us 
27983872
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_1_name 
short_term
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_1_power_limit_uw 
60000000
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_1_max_power_uw 
0
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_1_time_window_us 
2440
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_2_name 
peak_power
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_2_power_limit_uw 
105000000
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_2_max_power_uw 
0
localhost:~> cat /sys/class/powercap/intel-rapl/intel-rapl\:0/constraint_2_time_window_us 
cat: '/sys/class/powercap/intel-rapl/intel-rapl:0/constraint_2_time_window_us': No data available

None of those values line up with the observed behaviour.

Pheven42069 · November 9, 2024, 9:42am

Looks like the power limits and tau values are completely different.

PL2 lasts for way longer in Fedora and the limit is higher.

fedora:~$ sudo turbostat --Summary --quiet --show Busy%,Avg_MHz,PkgTmp,PkgWatt --interval 0.5
Avg_MHz	Busy%	PkgTmp	PkgWatt
33	2.65	48	2.55
140	5.12	49	4.28
140	5.37	49	3.71
93	5.46	49	3.56
71	5.36	48	3.40
165	9.24	48	4.61
22	2.00	48	2.08
20	1.85	48	2.11
778	21.12	72	12.26
3216	80.66	82	36.88
4001	99.77	84	44.70
4002	99.77	87	46.05
3989	99.72	83	44.80
3991	99.77	85	45.34
3952	99.66	86	45.17
3991	99.77	87	45.71
3990	99.75	87	46.04
3991	99.77	88	45.68
3990	99.75	88	46.50
3991	99.77	89	46.38
3991	99.77	90	46.09
3990	99.76	91	45.97
3982	99.77	92	46.38
4000	99.76	93	47.15
3991	99.77	94	47.09
3990	99.75	95	46.92
3991	99.77	95	47.41
3990	99.76	96	47.54
3991	99.77	97	48.13
3991	99.77	98	48.00
3991	99.77	99	48.13
3991	99.77	99	48.03
3987	99.67	100	47.80
3991	99.77	100	48.30
3978	99.77	100	48.32
3952	99.77	100	47.71
3925	99.71	100	46.73
3907	99.77	100	46.15
3893	99.75	100	45.54
3882	99.77	100	45.48
3857	99.71	100	44.89
3841	99.77	100	44.51
3846	99.77	100	44.09
3814	99.77	100	43.72
3805	99.77	100	43.33
3767	99.36	100	42.40
3766	99.77	100	41.94
3746	99.77	100	41.80
3722	99.77	100	41.01
3704	99.77	99	40.24
3705	99.69	100	40.22
3695	99.77	100	40.01
3341	99.77	86	32.26
2487	99.77	83	17.19
2432	99.77	82	16.50
2428	99.67	81	16.60
2439	99.77	81	16.59
2442	99.77	81	16.62
2454	99.77	81	16.77
2455	99.77	80	16.72
2459	99.77	81	16.77
2476	99.77	81	16.93
2469	99.61	79	16.89
2465	99.77	80	16.93
2475	99.77	80	16.94
2476	99.77	78	16.92
2486	99.77	78	16.98
2486	99.77	78	17.10
2486	99.77	78	17.02
2485	99.62	77	17.06
2491	99.77	78	17.09
2494	99.77	78	17.08
2494	99.77	77	17.10
2494	99.77	77	17.10
2494	99.77	77	17.06
2503	99.77	76	17.11

Any ideas why? I don’t have any idea what sets these values, I always thought the device manufacturer set them depending on how they’d designed the VRMs and cooling solution.

marel · November 10, 2024, 11:21am

Nice to see the s-tui pictures, I installed it and looks great but it give me not more insight.

What is good about this problem is that you have one OS running fine and the other OS showing the problem on the same computer which that solving the problem should be a matter of dumping all settings and comparing them. Easier said then done but you could start by running o both OS’s:

cd /sys/devices/system/cpu
find ./ -type f | xargs tail -n +1

Save that output to a file and compare the files for both OS’s.

heitormoreira · November 12, 2024, 3:37am

As a test, could you try to switch off “CPU Mitigations” on YaST bootloader?

system · December 12, 2024, 3:37am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.