SystemTap fails to find kernel tracepoints

I’m trying to learn how to use SystemTap, but I currently fail at even running the examples provided by the systemtap-docs package:

# stap /usr/share/doc/packages/systemtap/examples/process/schedtimes.stp
semantic error: while resolving probe point: identifier 'kernel' at /usr/share/doc/packages/systemtap/examples/process/schedtimes.stp:34:7
        source: probe kernel.trace("sched_switch") {
                      ^

semantic error: no match
semantic error: while resolving probe point: identifier 'kernel' at :105:7
        source: probe kernel.trace("sched_wakeup") {
                      ^

WARNING: cannot find module /tmp/stapmft4Pc/typequery_kmod_1/typequery_kmod_1.ko debuginfo: Unsupported relocation type
WARNING: cannot find module /root/.systemtap/cache/99/typequery_99c4493878436ab6976588e4e37d9622_708.ko debuginfo: Unsupported relocation type
Pass 2: analysis failed.  Try again with another '--vp 01' option.

Indeed, stap seems to indicate that there are no kernel tracepoints defined on my system:

# stap -L 'kernel.trace("*")'

returns nothing.

Generally, stap seems to be installed correctly on my openSUSE 12.2, as a simple check script runs as expected:

# stap -e 'probe vfs.read { printf("vfs.read
"); exit() }'
WARNING: cannot find module nfs debuginfo: Unsupported relocation type
WARNING: cannot find module sunrpc debuginfo: Unsupported relocation type
vfs.read

I can see the trace events in debugfs:

# ls -l /sys/kernel/debug/tracing/events/sched/sched_wakeup
total 0
-rw-r--r-- 1 root root 0 Jan 23 14:59 enable
-rw-r--r-- 1 root root 0 Jan 23 14:59 filter
-r--r--r-- 1 root root 0 Jan 23 14:59 format
-r--r--r-- 1 root root 0 Jan 23 14:59 id

My Google-fu fails me. Does anyone here have an idea what I could do to successfully use kernel tracepoints in SystemTap?

Thanks in advance!

On 01/23/2013 03:16 PM, braunseb wrote:
> Does anyone here have an idea what I could do to
> successfully use kernel tracepoints in SystemTap?

what version of systemtap?
from where did you fetch it and, how did you install it?

did you also install (for example) kernel-debuginfo,
kernel-debuginfo-common, and kernel-devel which exactly matches the
running kernel ?


dd
openSUSE®, the “German Engineered Automobile” of operating systems!

Hi, and thanks for answering!

I’m running 2.0.59, which I got from OBS (devel:tools repository) and installed via zypper.

I installed everything I could think of:

> sudo zypper se -i -s kernel systemtap
Loading repository data...
Reading installed packages...

S | Name                           | Type    | Version           | Arch   | Repository                
--+--------------------------------+---------+-------------------+--------+---------------------------
i | kernel-debug                   | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update      
i | kernel-debug-devel             | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update      
i | kernel-default-debuginfo       | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update-Debug
i | kernel-default-devel           | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update      
i | kernel-desktop                 | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update      
i | kernel-desktop-debuginfo       | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update-Debug
i | kernel-desktop-debugsource     | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update-Debug
i | kernel-desktop-devel           | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update      
i | kernel-desktop-devel-debuginfo | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update-Debug
i | kernel-devel                   | package | 3.4.11-2.16.1     | noarch | openSUSE-12.2-Update      
i | kernel-firmware                | package | 20120719git-2.9.1 | noarch | openSUSE-12.2-Update      
i | kernel-source                  | package | 3.4.11-2.16.1     | noarch | openSUSE-12.2-Update      
i | kernel-syms                    | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update      
i | kernel-trace                   | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update      
i | kernel-trace-debuginfo         | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update-Debug
i | kernel-trace-debugsource       | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update-Debug
i | kernel-trace-devel             | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update      
i | kernel-trace-devel-debuginfo   | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update-Debug
i | kernel-xen-devel               | package | 3.4.11-2.16.1     | x86_64 | openSUSE-12.2-Update      
i | systemtap                      | package | 2.0-59.2          | x86_64 | devel:tools               
i | systemtap-docs                 | package | 2.0-60.4          | noarch | devel:tools               
i | systemtap-runtime              | package | 2.0-59.2          | x86_64 | devel:tools               
i | systemtap-sdt-devel            | package | 2.0-59.2          | x86_64 | devel:tools               
i | systemtap-server               | package | 2.0-59.2          | x86_64 | devel:tools

The versions should match, too:

> uname -r
3.4.11-2.16-trace

software.opensuse.org does not know about kernel-debuginfo-common, and I can’t find it with zypper se. What else am I missing?

On 01/23/2013 04:26 PM, braunseb wrote:
>
> I’m running 2.0.59, which I got from OBS (devel:tools repository) and
> installed via zypper.

do as you wish, but i believe if you were to install the version
available for 12.2 it might run okay…

http://software.opensuse.org/package/systemtap shows that to be 1.7, i’d
guess (i don’t have a 12.2 to try) it is in oss or non-oss and should
‘just work’ if installed with YaST Software Management or “zypper in”
from the standard repo (OBS is a different kettle of fish)

of course, i’d begin by uninstalling the version from OBS.

> software.opensuse.org does not know about kernel-debuginfo-common, and
> I can’t find it with zypper se. What else am I missing?

hmmmmmm…i don’t know, i googled to
http://sourceware.org/systemtap/SystemTap_Beginners_Guide/using-systemtap.html#using-setup
and found some notes that are probably far too specific to Fedora (that
happens sometimes, sorry)…

oh! try picking up missing specific hints here (and note the use of
google’s “site specifier” to force the search inside the openSUSE
universe, which i should’a done the first time):
https://www.google.com/search?q=site%3Aopensuse.org+installing+systemtap
https://www.google.com/search?q=site%3Aopensuse.org+systemtap
https://www.google.com/search?q=systemtap+suse


dd
openSUSE®, the “German Engineered Automobile” of operating systems!
http://goo.gl/PUjnL

Please run stap-report, and stap -vvv [other options], and send their outputs to systemtap@sourceware.org.

Just tried downgrading to 1.7, and confirmed that it generally works:

> zypper se -s -i systemtap
Loading repository data...
Reading installed packages...

S | Name              | Type    | Version   | Arch   | Repository          
--+-------------------+---------+-----------+--------+---------------------
i | systemtap         | package | 1.7-3.4.1 | x86_64 | openSUSE-12.2-Update
i | systemtap-runtime | package | 1.7-3.4.1 | x86_64 | openSUSE-12.2-Update

> stap --version
Systemtap translator/driver (version 1.7/0.153 non-git sources)
Copyright (C) 2005-2012 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
enabled features: LIBSQLITE3 NSS TR1_UNORDERED_MAP NLS

> sudo stap -e 'probe vfs.read { printf("Works.
"); exit() }'
WARNING: cannot find module nfs debuginfo: Unsupported relocation type
WARNING: cannot find module sunrpc debuginfo: Unsupported relocation type
Works.

But still no luck with tracepoints:


> sudo stap -L 'kernel.trace("*")'
>

oh! try picking up missing specific hints here (and note the use of
google’s “site specifier” to force the search inside the openSUSE
universe, which i should’a done the first time):
site:opensuse.org installing systemtap - Google Suche
site:opensuse.org systemtap - Google Suche
systemtap suse - Google Suche

I actually tried that before posting and found a lot of examples where someone was able to get systemtap to run for other probes or the canonical vfs.read example.
And indeed systemtap works for a bunch of other probes I tried. I also tried site:opensuse.org systemtap "kernel.trace" - Google Suche, but that did not yield any information pertaining to my problem.

When I try running stap -vvvv, it provides a hint:


> sudo stap -vvvv -L 'kernel.trace("*")'
Systemtap translator/driver (version 1.7/0.153 non-git sources)
Copyright (C) 2005-2012 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
enabled features: LIBSQLITE3 NSS TR1_UNORDERED_MAP NLS
Created temporary directory "/tmp/stap9cJQKh"
Session arch: x86_64 release: 3.4.11-2.16-trace
Parsed kernel "/lib/modules/3.4.11-2.16-trace/build/.config", containing 4484 tuples
Parsed kernel /lib/modules/3.4.11-2.16-trace/build/Module.symvers, which contained 5890 vmlinux exports
Searched: " /usr/share/systemtap/tapset/x86_64/*.stp ", found: 7, processed: 7
Searched: " /usr/share/systemtap/tapset/*.stp ", found: 77, processed: 77
Pass 1: parsed user script and 84 library script(s) using 76904virt/27072res/2540shr kb, in 100usr/20sys/125real ms.
Attempting to extract kernel debuginfo build ID from /lib/modules/3.4.11-2.16-trace/build/vmlinux.id
Attempting to extract kernel debuginfo build ID from /sys/kernel/notes
Located kernel source tree (DW_AT_comp_dir) at '/usr/src/debug//////////////kernel-trace-3.4.11/linux-obj'
Checking tracepoint glob /lib/modules/3.4.11-2.16-trace/build/include/trace/events/*.h
Checking tracepoint glob /lib/modules/3.4.11-2.16-trace/build/include/trace/*.h
Checking tracepoint glob /lib/modules/3.4.11-2.16-trace/build/arch/x86/kvm/*trace.h
Checking tracepoint glob /lib/modules/3.4.11-2.16-trace/build/fs/xfs/linux-*/xfs_tr*.h
Checking tracepoint glob /usr/src/debug//////////////kernel-trace-3.4.11/linux-obj/include/trace/events/*.h
Checking tracepoint glob /usr/src/debug//////////////kernel-trace-3.4.11/linux-obj/include/trace/*.h
Checking tracepoint glob /usr/src/debug//////////////kernel-trace-3.4.11/linux-obj/arch/x86/kvm/*trace.h
Checking tracepoint glob /usr/src/debug//////////////kernel-trace-3.4.11/linux-obj/fs/xfs/linux-*/xfs_tr*.h
Pass 2: getting a tracepoint query for 0 headers: 
blacklist regexps:
blfn: ^(atomic_notifier_call_chain|default_do_nmi|__die|die_nmi|do_debug|do_general_protection|do_int3|do_IRQ|do_page_fault|do_sparc64_fault|do_trap|dummy_nmi_callback|flush_icache_range|ia64_bad_break|ia64_do_page_fault|ia64_fault|io_check_error|mem_parity_error|nmi_watchdog_tick|notifier_call_chain|oops_begin|oops_end|program_check_exception|single_step_exception|sync_regs|unhandled_fault|unknown_nmi_error|xen_[gs]et_debugreg|xen_irq_.*|xen_.*_fl_direct.*|check_events|xen_adjust_exception_frame|xen_iret.*|xen_sysret64.*|test_ti_thread_flag.*|inat_get_opcode_attribute|system_call_after_swapgs|.*raw_.*_lock.*|.*raw_.*_unlock.*|.*raw_.*_trylock.*|.*read_lock.*|.*read_unlock.*|.*read_trylock.*|.*write_lock.*|.*write_unlock.*|.*write_trylock.*|.*write_seqlock.*|.*write_sequnlock.*|.*spin_lock.*|.*spin_unlock.*|.*spin_trylock.*|.*spin_is_locked.*|rwsem_.*lock.*|.*mutex_.*lock.*|raw_.*|atomic_.*|atomic64_.*|get_bh|put_bh|.*apic.*|.*APIC.*|.*softirq.*|.*IRQ.*|.*_intr.*|__delay|.*kernel_text.*|get_current|current_.*|.*exception_tables.*|.*setup_rt_frame.*|.*preempt_count.*|preempt_schedule|__switch_to|special_mapping_.*|.*_pte_.*)$
blfn_ret: ^(do_exit|sys_exit|sys_exit_group)$
blfile: ^(kernel/kprobes\.c|arch/.*/kernel/kprobes\.c|.*/include/asm/io\.h|.*/include/asm/io_64\.h|.*/include/asm/bitops\.h|drivers/ide/ide-iops\.c|arch/.*/kernel/paravirt\.c|.*/include/asm/paravirt\.h|fs/seq_file\.c)$
blsection: ^(\.init\.|\.exit\.|\.devinit\.|\.devexit\.|\.cpuinit\.|\.cpuexit\.|\.meminit\.|\.memexit\.)
pattern '*' matches module 'kernel'
semantic error: no match while resolving probe point kernel.trace("*")
deleting module_cache
tracepoint_builder releasing dwflpp
Pass 2: analyzed script: 0 probe(s), 0 function(s), 0 embed(s), 0 global(s) using 204828virt/28504res/2880shr kb, in 10usr/60sys/72real ms.
Running rm -rf /tmp/stap9cJQKh
Spawn waitpid result (0x0): 0

and it looks to me like the lines with header globs show that there is a problem finding the kernel include files (they are in /usr/src/debug/kernel-trace-3.4.11/linux-3.4 on my machine).
Symlinking them to the location where SystemTap looks for them does yield a change in that stap actually compiles a kernel module, but the error condition remains:


> sudo stap -vvvv -L 'kernel.trace("*")'
SNIP lots of compilation messages
semantic error: no match while resolving probe point kernel.trace("*")
SNIP
... full output at [stap -vvvv -L 'kernel.trace("*")' - Pastebin.com](http://pastebin.com/7QZqzpwB)

stap-report’s output is at stap-report - Pastebin.com, for the record.

On 01/24/2013 09:16 AM, braunseb wrote:
>
> > sudo stap -e ‘probe vfs.read { printf("Works.
"); exit() }’
> > sudo stap -L ‘kernel.trace("")’
> > sudo stap -vvvv -L 'kernel.trace("
")’
> > sudo stap -vvvv -L ‘kernel.trace("*")’

it may not be helpful in this case, but i think it important for you to
know that ‘sudo [someExecutable]’ is not the same as being root and
running [someExecutable]…

so, just try


su -c [someExecutable]
like:
su -c stap -e | -L | -vvv | whatever]

as said, it might make a difference, it might not…it depends because
root’s PATH is not the same as your PATH using sudo…and if you must
have root’s PATH to be successful you will fail if using sudo (in
openSUSE…Debian–i have no idea.)

more info:
http://tinyurl.com/593e4c
http://tinyurl.com/ydbwssh
http://tinyurl.com/4nsaqst
http://tinyurl.com/665h5ek
http://www.linfo.org/root.html


dd
openSUSE®, the “German Engineered Automobile” of operating systems!
http://tinyurl.com/DD-Caveat

I tend to avoid su -c, because su -c accepts a single argument, forcing you to add an additional layer of quotes, so actually

su -c stap -L 'kernel.trace("*")'

wouldn’t work, but

su -c "stap -L 'kernel.trace(\"*\")'"

would (be sure to get the quotes right, though).

In this case, I don’t think it’s a problem with the PATH. Just to be sure, I ran it again from a root login session, but no change. Also, stap is supposed to run equally well if the calling user is a member of the stapdev or stapsys groups, so you (theoretically) should not need to be root anyway.

Okay, I managed to figure it out:

SystemTap looks for tracepoints by compiling a fake kernel module and looking at the debug information that is generated, using the elfutils libraries libelf1 and libdw1. These in turn depend on the backends that are packaged in libebl1 to actually resolve the relocations in the debug info, so if libebl1 is not installed, the whole process fails. After installing libebl1, systemtap does find the tracepoints.

I guess I’ll file a bug against systemtap.

On 01/24/2013 12:26 PM, braunseb wrote:
> After installing libebl1, systemtap does find the tracepoints.

happy you found the problem…just wondering: did you find it because
it finally popped up in an error message, or ???

> I guess I’ll file a bug against systemtap.

i’d guess it best to file that in our bugzilla, here
http://tinyurl.com/nzhq7j

because it might be an openSUSE packaging problem (failed to include all
dependencies)…

once you have bug number/URL please return that info to this thread (so
any who google in with the same problem can look to the bug to find work
around and/or follow the progress of the resolution)

tia


dd

happy you found the problem…just wondering: did you find it because
it finally popped up in an error message, or ???

I remembered seeing a warning message about unsupported relocations which I did not associate with this behavior, but I can’t remember where it said so. It finally started bugging me in the back of my mind, so I started investigating.

i’d guess it best to file that in our bugzilla, here
openSUSE:Submitting bug reports - openSUSE Wiki

because it might be an openSUSE packaging problem (failed to include all
dependencies)…

Clarification: That’s what I originally meant. Bug report can be found at Access Denied

Thanks for your help!

Well, actually it was in the very first log messages I quoted in this thread. This is embarrassing.

> This is embarrassing.

don’t be…its complicated.
thanks for your bug.


dd
openSUSE®, the “German Engineered Automobile” of operating systems!

Just wanted to thank you for this hint. I had the same problem, and hit your post, and created a symlink too. It worked like a whizz! And no kernel module compilation errors too. (I have libebl1 installed, though I am not sure if I had installed that on my own or was installed as a dependency automatically. I was trying out various means to get this working, in the course of time, I installed many, many packages all claiming to solve this problem, but never actually did. Until I read this hint of yours.

Thanks!