2015-12-29 20:41:04 +00:00
|
|
|
# Linux Profiling
|
|
|
|
|
|
|
|
Profiling code is enabled when the `use_allocator` variable in gyp is set to
|
build: Stop using hermetic clang, libc++, etc
This brings our default build config more in line with what is
necessary for some platforms anyway: using the system-installed
toolchain and sysroot to build everything.
We will no longer fetch source or binaries for any specific build
tools, such as libc++, clang, gold, binutils, or valgrind.
The main part of this change is the changing of default gyp settings
in gyp_packager.py. For this, a bug in gyp_packager.py had to be
fixed, in which similar GYP_DEFINE key names (such as clang and
host_clang) would conflict, causing some defaults not to be installed
properly.
In order to enable clang=0 by default, some changes had to be made in
common.gypi:
- compiler macros added to fix a compatibility issue between
Chromium's base/mac/ folder and the actual OSX SDK
- replaced clang_warning_flags variables with standard cflags
settings, plus xcode_settings for OSX
- turned off warnings-as-errors for non-shaka code, rather than
allow-listing specific warning types, since we can't actually fix
those warnings on any platform
- disabled two specific warnings in shaka code, both of which are
caused by headers from our non-shaka dependencies
Also, one warning (missing "override" keyword) has been fixed in
vod_media_info_dump_muxer_listener.h.
Although these changes were done to make building simpler on a wider
array of platforms (arm64, for example), it seems to make the build a
bit faster, too. For me, at least, on my main Linux workstation:
- "gclient sync" now runs 20-30% faster
- "ninja -C out/Release" now runs 5-13% faster
The following environment variables are no longer required:
- DEPOT_TOOLS_WIN_TOOLCHAIN
- MACOSX_DEPLOYMENT_TARGET
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following GYP_DEFINES are no longer required for anyone:
- clang=0
- host_clang=0
- clang_xcode=1
- use_allocator=none
- use_experimental_allocator_shim=0
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following repos are no longer dependencies in gclient:
- binutils
- clang
- gold
- libc++
- libc++abi
- valgrind
The following gclient hooks have been removed:
- clang
- mac_toolchain
- sysroot
Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
|
|
|
`tcmalloc` and `profiling` variable in gyp is set to `1`. That will build the
|
|
|
|
tcmalloc library, including the cpu profiling and heap profiling code into
|
|
|
|
shaka-packager, e.g.
|
2015-12-29 20:41:04 +00:00
|
|
|
|
|
|
|
GYP_DEFINES='profiling=1 use_allocator="tcmalloc"' gclient runhooks
|
|
|
|
|
|
|
|
If the stack traces in your profiles are incomplete, this may be due to missing
|
|
|
|
frame pointers in some of the libraries. A workaround is to use the
|
|
|
|
`linux_keep_shadow_stacks=1` gyp option. This will keep a shadow stack using the
|
|
|
|
`-finstrument-functions` option of gcc and consult the stack when unwinding.
|
|
|
|
|
|
|
|
## CPU Profiling
|
|
|
|
|
2016-05-17 00:41:02 +00:00
|
|
|
In order to enable cpu profiling, run shaka-packager with the environment
|
2015-12-29 20:41:04 +00:00
|
|
|
variable `CPUPROFILE` set to a filename. For example:
|
|
|
|
|
|
|
|
CPUPROFILE=/tmp/cpuprofile out/Release/packager
|
|
|
|
|
|
|
|
The cpu profile will be dumped periodically to the filename specified in the
|
|
|
|
CPUPROFILE environment variable. You can then analyze the dumps using the pprof
|
2018-03-12 17:15:47 +00:00
|
|
|
script (`packager/third_party/tcmalloc/chromium/src/pprof`). For example,
|
2015-12-29 20:41:04 +00:00
|
|
|
|
|
|
|
pprof --gv out/Release/packager /tmp/cpuprofile
|
|
|
|
|
|
|
|
This will generate a visual representation of the cpu profile as a postscript
|
|
|
|
file and load it up using `gv`. For more powerful commands, please refer to the
|
|
|
|
pprof help output and the google-perftools documentation.
|
|
|
|
|
|
|
|
For further information, please refer to
|
|
|
|
http://google-perftools.googlecode.com/svn/trunk/doc/cpuprofile.html.
|
|
|
|
|
|
|
|
## Heap Profiling
|
|
|
|
|
2016-05-17 00:41:02 +00:00
|
|
|
To turn on the heap profiler on shaka-packager, use the `HEAPPROFILE`
|
2015-12-29 20:41:04 +00:00
|
|
|
environment variable to specify a filename for the heap profile. For example:
|
|
|
|
|
|
|
|
HEAPPROFILE=/tmp/heapprofile out/Release/packager
|
|
|
|
|
|
|
|
The heap profile will be dumped periodically to the filename specified in the
|
|
|
|
`HEAPPROFILE` environment variable. The dumps can be analyzed using the same
|
|
|
|
command as cpu profiling above.
|
|
|
|
|
|
|
|
For further information, please refer to
|
|
|
|
http://google-perftools.googlecode.com/svn/trunk/doc/heapprofile.html.
|
|
|
|
|
|
|
|
Some tests fork short-living processes which have a small memory footprint. To
|
|
|
|
catch those, use the `HEAP_PROFILE_ALLOCATION_INTERVAL` environment variable.
|
|
|
|
|
|
|
|
#### Dumping a profile of a running process
|
|
|
|
|
|
|
|
To programmatically generate a heap profile before exit, use code like:
|
|
|
|
|
|
|
|
#include "packager/third_party/tcmalloc/chromium/src/gperftools/heap-profiler.h"
|
|
|
|
|
|
|
|
// "foobar" will be included in the message printed to the console
|
|
|
|
HeapProfilerDump("foobar");
|
|
|
|
|
|
|
|
Then add allocator.gyp dependency to the target with the above change:
|
|
|
|
|
|
|
|
'conditions': [
|
|
|
|
['profiling==1', {
|
|
|
|
'dependencies': [
|
|
|
|
'base/allocator/allocator.gyp:allocator',
|
|
|
|
],
|
|
|
|
}],
|
|
|
|
],
|
|
|
|
|
|
|
|
Or you can use gdb to attach at any point:
|
|
|
|
|
|
|
|
1. Attach gdb to the process: `$ gdb -p 12345`
|
|
|
|
2. Cause it to dump a profile: `(gdb) p HeapProfilerDump("foobar")`
|
|
|
|
3. The filename will be printed on the console, e.g.
|
|
|
|
"`Dumping heap profile to heap.0001.heap (foobar)`"
|
|
|
|
|
build: Stop using hermetic clang, libc++, etc
This brings our default build config more in line with what is
necessary for some platforms anyway: using the system-installed
toolchain and sysroot to build everything.
We will no longer fetch source or binaries for any specific build
tools, such as libc++, clang, gold, binutils, or valgrind.
The main part of this change is the changing of default gyp settings
in gyp_packager.py. For this, a bug in gyp_packager.py had to be
fixed, in which similar GYP_DEFINE key names (such as clang and
host_clang) would conflict, causing some defaults not to be installed
properly.
In order to enable clang=0 by default, some changes had to be made in
common.gypi:
- compiler macros added to fix a compatibility issue between
Chromium's base/mac/ folder and the actual OSX SDK
- replaced clang_warning_flags variables with standard cflags
settings, plus xcode_settings for OSX
- turned off warnings-as-errors for non-shaka code, rather than
allow-listing specific warning types, since we can't actually fix
those warnings on any platform
- disabled two specific warnings in shaka code, both of which are
caused by headers from our non-shaka dependencies
Also, one warning (missing "override" keyword) has been fixed in
vod_media_info_dump_muxer_listener.h.
Although these changes were done to make building simpler on a wider
array of platforms (arm64, for example), it seems to make the build a
bit faster, too. For me, at least, on my main Linux workstation:
- "gclient sync" now runs 20-30% faster
- "ninja -C out/Release" now runs 5-13% faster
The following environment variables are no longer required:
- DEPOT_TOOLS_WIN_TOOLCHAIN
- MACOSX_DEPLOYMENT_TARGET
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following GYP_DEFINES are no longer required for anyone:
- clang=0
- host_clang=0
- clang_xcode=1
- use_allocator=none
- use_experimental_allocator_shim=0
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following repos are no longer dependencies in gclient:
- binutils
- clang
- gold
- libc++
- libc++abi
- valgrind
The following gclient hooks have been removed:
- clang
- mac_toolchain
- sysroot
Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
|
|
|
|
|
|
|
## Thread sanitizer (tsan)
|
|
|
|
|
|
|
|
To compile with the thread sanitizer library (tsan), you must set clang as your
|
|
|
|
compiler and set the `tsan=1` and `tsan_blacklist` configs:
|
|
|
|
|
|
|
|
CC=clang CXX=clang++ GYP_DEFINES="tsan=1 tsan_blacklist=/path/to/src/packager/tools/memory/tsan_v2/ignores.txt" gclient runhooks
|
|
|
|
|
|
|
|
NOTE: tsan and asan cannot be used at the same time.
|
|
|
|
|
|
|
|
|
|
|
|
## Adddress sanitizer (asan)
|
|
|
|
|
|
|
|
To compile with the address sanitizer library (asan), you must set clang as your
|
|
|
|
compiler and set the `asan=1` config:
|
|
|
|
|
|
|
|
CC=clang CXX=clang++ GYP_DEFINES="asan=1" gclient runhooks
|
|
|
|
|
|
|
|
NOTE: tsan and asan cannot be used at the same time.
|
|
|
|
|
|
|
|
|
|
|
|
## Leak sanitizer (lsan)
|
|
|
|
|
|
|
|
To compile with the leak sanitizer library (lsan), you must set clang as your
|
|
|
|
compiler and set the `lsan=1` config:
|
|
|
|
|
|
|
|
CC=clang CXX=clang++ GYP_DEFINES="lsan=1" gclient runhooks
|
|
|
|
|
|
|
|
|
2015-12-29 20:41:04 +00:00
|
|
|
## Reference
|
|
|
|
|
|
|
|
[Linux Profiling in Chromium](https://chromium.googlesource.com/chromium/src/+/master/docs/linux_profiling.md)
|