41.5. Analyzing the Data
Periodically, the OProfile daemon, oprofiled, collects the samples and writes them to the /var/lib/oprofile/samples/ directory. Before reading the data, make sure all data has been written to this directory by executing the following command as root:
opcontrol --dump |
Each sample file name is based on the name of the executable. For example, the samples for the default event on a Pentium III processor for /bin/bash becomes:
\{root\}/bin/bash/\{dep\}/\{root\}/bin/bash/CPU_CLK_UNHALTED.100000 |
The following tools are available to profile the sample data once it has been collected:
opreport
opannotate
Use these tools, along with the binaries profiled, to generate reports that can be further analyzed.
 | Warning |
|---|---|
The executable being profiled must be used with these tools to analyze the data. If it must change after the data is collected, backup the executable used to create the samples as well as the sample files. |
Samples for each executable are written to a single sample file. Samples from each dynamically linked library are also written to a single sample file. While OProfile is running, if the executable being monitored changes and a sample file for the executable exists, the existing sample file is automatically deleted. Thus, if the existing sample file is needed, it must be backed up, along with the executable used to create it before replacing the executable with a new version. Refer to Section 41.4 Saving Data for details on how to backup the sample file.
41.5.1. Using opreport
The opreport tool provides an overview of all the executables being profiled.
The following is part of an example output:
Profiling through timer interrupt
TIMER:0|
samples| %|
------------------
25926 97.5212 no-vmlinux
359 1.3504 pi
65 0.2445 Xorg
62 0.2332 libvte.so.4.4.0
56 0.2106 libc-2.3.4.so
34 0.1279 libglib-2.0.so.0.400.7
19 0.0715 libXft.so.2.1.2
17 0.0639 bash
8 0.0301 ld-2.3.4.so
8 0.0301 libgdk-x11-2.0.so.0.400.13
6 0.0226 libgobject-2.0.so.0.400.7
5 0.0188 oprofiled
4 0.0150 libpthread-2.3.4.so
4 0.0150 libgtk-x11-2.0.so.0.400.13
3 0.0113 libXrender.so.1.2.2
3 0.0113 du
1 0.0038 libcrypto.so.0.9.7a
1 0.0038 libpam.so.0.77
1 0.0038 libtermcap.so.2.0.8
1 0.0038 libX11.so.6.2
1 0.0038 libgthread-2.0.so.0.400.7
1 0.0038 libwnck-1.so.4.9.0 |
Each executable is listed on its own line. The first column is the number of samples recorded for the executable. The second column is the percentage of samples relative to the total number of samples. The third column is the name of the executable.
Refer to the opreport man page for a list of available command line options, such as the -r option used to sort the output from the executable with the smallest number of samples to the one with the largest number of samples.
41.5.2. Using opreport on a Single Executable
To retrieve more detailed profiled information about a specific executable, use opreport:
opreport <mode> <executable> |
<executable> must be the full path to the executable to be analyzed. <mode> must be one of the following:
- -l
List sample data by symbols. For example, the following is part of the output from running the command opreport -l /lib/tls/libc-<version>.so:
samples % symbol name 12 21.4286 __gconv_transform_utf8_internal 5 8.9286 _int_malloc 4 7.1429 malloc 3 5.3571 __i686.get_pc_thunk.bx 3 5.3571 _dl_mcount_wrapper_check 3 5.3571 mbrtowc 3 5.3571 memcpy 2 3.5714 _int_realloc 2 3.5714 _nl_intern_locale_data 2 3.5714 free 2 3.5714 strcmp 1 1.7857 __ctype_get_mb_cur_max 1 1.7857 __unregister_atfork 1 1.7857 __write_nocancel 1 1.7857 _dl_addr 1 1.7857 _int_free 1 1.7857 _itoa_word 1 1.7857 calc_eclosure_iter 1 1.7857 fopen@@GLIBC_2.1 1 1.7857 getpid 1 1.7857 memmove 1 1.7857 msort_with_tmp 1 1.7857 strcpy 1 1.7857 strlen 1 1.7857 vfprintf 1 1.7857 write
The first column is the starting virtual memory address (vma). The second column is the number of samples for the symbol. The third column is the percentage of samples for this symbol relative to the overall samples for the executable, and the fourth column is the symbol name.
To sort the output from the largest number of samples to the smallest (reverse order), use -r in conjunction with the -l option.
- -i <symbol-name>
List sample data specific to a symbol name. For example, the following output is from the command opreport -l -i __gconv_transform_utf8_internal /lib/tls/libc-<version>.so:
samples % symbol name 12 100.000 __gconv_transform_utf8_internal
The first line is a summary for the symbol/executable combination.
The first column is the number of samples for the memory symbol. The second column is the percentage of samples for the memory address relative to the total number of samples for the symbol. The third column is the symbol name.
- -d
List sample data by symbols with more detail than -l. For example, the following output is from the command opreport -l -d __gconv_transform_utf8_internal /lib/tls/libc-<version>.so:
vma samples % symbol name 00a98640 12 100.000 __gconv_transform_utf8_internal 00a98640 1 8.3333 00a9868c 2 16.6667 00a9869a 1 8.3333 00a986c1 1 8.3333 00a98720 1 8.3333 00a98749 1 8.3333 00a98753 1 8.3333 00a98789 1 8.3333 00a98864 1 8.3333 00a98869 1 8.3333 00a98b08 1 8.3333
The data is the same as the -l option except that for each symbol, each virtual memory address used is shown. For each virtual memory address, the number of samples and percentage of samples relative to the number of samples for the symbol is displayed.
- -x <symbol-name>
Exclude the comma-separated list of symbols from the output.
- session:<name>
Specify the full path to the session or a directory relative to the /var/lib/oprofile/samples/ directory.
41.5.3. Using opannotate
The opannotate tool tries to match the samples for particular instructions to the corresponding lines in the source code. The resulting files generated should have the samples for the lines at the left. It also puts in a comment at the beginning of each function listing the total samples for the function.
For this utility to work, the executable must be compiled with GCC's -g option. By default, Red Hat Enterprise Linux packages are not compiled with this option.
The general syntax for opannotate is as follows:
opannotate --search-dirs <src-dir> --source <executable> |
The directory containing the source code and the executable to be analyzed must be specified. Refer to the opannotate man page for a list of additional command line options.