2015-12-29 20:41:04 +00:00
|
|
|
# Linux Profiling
|
|
|
|
|
2023-12-01 17:32:19 +00:00
|
|
|
In theory we should be able to build packager using
|
|
|
|
[gperftools](https://github.com/gperftools/gperftools/tree/master) to
|
|
|
|
get back the profiling functionality described below. However actually
|
|
|
|
integrating this into the CMake build is not yet done. Pull requests
|
|
|
|
welcome. See https://github.com/shaka-project/shaka-packager/issues/1277
|
2015-12-29 20:41:04 +00:00
|
|
|
|
2023-12-01 17:32:19 +00:00
|
|
|
If packager was linked using `-ltcmalloc` then the following
|
|
|
|
instructions should work:
|
2015-12-29 20:41:04 +00:00
|
|
|
|
|
|
|
## CPU Profiling
|
|
|
|
|
2016-05-17 00:41:02 +00:00
|
|
|
In order to enable cpu profiling, run shaka-packager with the environment
|
2015-12-29 20:41:04 +00:00
|
|
|
variable `CPUPROFILE` set to a filename. For example:
|
|
|
|
|
|
|
|
CPUPROFILE=/tmp/cpuprofile out/Release/packager
|
|
|
|
|
|
|
|
The cpu profile will be dumped periodically to the filename specified in the
|
|
|
|
CPUPROFILE environment variable. You can then analyze the dumps using the pprof
|
2018-03-12 17:15:47 +00:00
|
|
|
script (`packager/third_party/tcmalloc/chromium/src/pprof`). For example,
|
2015-12-29 20:41:04 +00:00
|
|
|
|
|
|
|
pprof --gv out/Release/packager /tmp/cpuprofile
|
|
|
|
|
|
|
|
This will generate a visual representation of the cpu profile as a postscript
|
|
|
|
file and load it up using `gv`. For more powerful commands, please refer to the
|
|
|
|
pprof help output and the google-perftools documentation.
|
|
|
|
|
|
|
|
For further information, please refer to
|
|
|
|
http://google-perftools.googlecode.com/svn/trunk/doc/cpuprofile.html.
|
|
|
|
|
|
|
|
## Heap Profiling
|
|
|
|
|
2016-05-17 00:41:02 +00:00
|
|
|
To turn on the heap profiler on shaka-packager, use the `HEAPPROFILE`
|
2015-12-29 20:41:04 +00:00
|
|
|
environment variable to specify a filename for the heap profile. For example:
|
|
|
|
|
|
|
|
HEAPPROFILE=/tmp/heapprofile out/Release/packager
|
|
|
|
|
|
|
|
The heap profile will be dumped periodically to the filename specified in the
|
|
|
|
`HEAPPROFILE` environment variable. The dumps can be analyzed using the same
|
|
|
|
command as cpu profiling above.
|
|
|
|
|
|
|
|
For further information, please refer to
|
|
|
|
http://google-perftools.googlecode.com/svn/trunk/doc/heapprofile.html.
|
|
|
|
|
|
|
|
Some tests fork short-living processes which have a small memory footprint. To
|
|
|
|
catch those, use the `HEAP_PROFILE_ALLOCATION_INTERVAL` environment variable.
|
|
|
|
|
|
|
|
#### Dumping a profile of a running process
|
|
|
|
|
|
|
|
To programmatically generate a heap profile before exit, use code like:
|
|
|
|
|
2023-12-01 17:32:19 +00:00
|
|
|
#include <gperftools/heap-profiler.h>
|
2015-12-29 20:41:04 +00:00
|
|
|
|
|
|
|
// "foobar" will be included in the message printed to the console
|
|
|
|
HeapProfilerDump("foobar");
|
|
|
|
|
|
|
|
Or you can use gdb to attach at any point:
|
|
|
|
|
|
|
|
1. Attach gdb to the process: `$ gdb -p 12345`
|
|
|
|
2. Cause it to dump a profile: `(gdb) p HeapProfilerDump("foobar")`
|
|
|
|
3. The filename will be printed on the console, e.g.
|
|
|
|
"`Dumping heap profile to heap.0001.heap (foobar)`"
|
|
|
|
|
build: Stop using hermetic clang, libc++, etc
This brings our default build config more in line with what is
necessary for some platforms anyway: using the system-installed
toolchain and sysroot to build everything.
We will no longer fetch source or binaries for any specific build
tools, such as libc++, clang, gold, binutils, or valgrind.
The main part of this change is the changing of default gyp settings
in gyp_packager.py. For this, a bug in gyp_packager.py had to be
fixed, in which similar GYP_DEFINE key names (such as clang and
host_clang) would conflict, causing some defaults not to be installed
properly.
In order to enable clang=0 by default, some changes had to be made in
common.gypi:
- compiler macros added to fix a compatibility issue between
Chromium's base/mac/ folder and the actual OSX SDK
- replaced clang_warning_flags variables with standard cflags
settings, plus xcode_settings for OSX
- turned off warnings-as-errors for non-shaka code, rather than
allow-listing specific warning types, since we can't actually fix
those warnings on any platform
- disabled two specific warnings in shaka code, both of which are
caused by headers from our non-shaka dependencies
Also, one warning (missing "override" keyword) has been fixed in
vod_media_info_dump_muxer_listener.h.
Although these changes were done to make building simpler on a wider
array of platforms (arm64, for example), it seems to make the build a
bit faster, too. For me, at least, on my main Linux workstation:
- "gclient sync" now runs 20-30% faster
- "ninja -C out/Release" now runs 5-13% faster
The following environment variables are no longer required:
- DEPOT_TOOLS_WIN_TOOLCHAIN
- MACOSX_DEPLOYMENT_TARGET
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following GYP_DEFINES are no longer required for anyone:
- clang=0
- host_clang=0
- clang_xcode=1
- use_allocator=none
- use_experimental_allocator_shim=0
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following repos are no longer dependencies in gclient:
- binutils
- clang
- gold
- libc++
- libc++abi
- valgrind
The following gclient hooks have been removed:
- clang
- mac_toolchain
- sysroot
Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
|
|
|
|
|
|
|
## Thread sanitizer (tsan)
|
|
|
|
|
|
|
|
To compile with the thread sanitizer library (tsan), you must set clang as your
|
2023-12-01 17:32:19 +00:00
|
|
|
compiler and set `-fsanitize=thread` in compiler flags.
|
build: Stop using hermetic clang, libc++, etc
This brings our default build config more in line with what is
necessary for some platforms anyway: using the system-installed
toolchain and sysroot to build everything.
We will no longer fetch source or binaries for any specific build
tools, such as libc++, clang, gold, binutils, or valgrind.
The main part of this change is the changing of default gyp settings
in gyp_packager.py. For this, a bug in gyp_packager.py had to be
fixed, in which similar GYP_DEFINE key names (such as clang and
host_clang) would conflict, causing some defaults not to be installed
properly.
In order to enable clang=0 by default, some changes had to be made in
common.gypi:
- compiler macros added to fix a compatibility issue between
Chromium's base/mac/ folder and the actual OSX SDK
- replaced clang_warning_flags variables with standard cflags
settings, plus xcode_settings for OSX
- turned off warnings-as-errors for non-shaka code, rather than
allow-listing specific warning types, since we can't actually fix
those warnings on any platform
- disabled two specific warnings in shaka code, both of which are
caused by headers from our non-shaka dependencies
Also, one warning (missing "override" keyword) has been fixed in
vod_media_info_dump_muxer_listener.h.
Although these changes were done to make building simpler on a wider
array of platforms (arm64, for example), it seems to make the build a
bit faster, too. For me, at least, on my main Linux workstation:
- "gclient sync" now runs 20-30% faster
- "ninja -C out/Release" now runs 5-13% faster
The following environment variables are no longer required:
- DEPOT_TOOLS_WIN_TOOLCHAIN
- MACOSX_DEPLOYMENT_TARGET
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following GYP_DEFINES are no longer required for anyone:
- clang=0
- host_clang=0
- clang_xcode=1
- use_allocator=none
- use_experimental_allocator_shim=0
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following repos are no longer dependencies in gclient:
- binutils
- clang
- gold
- libc++
- libc++abi
- valgrind
The following gclient hooks have been removed:
- clang
- mac_toolchain
- sysroot
Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
|
|
|
|
|
|
|
NOTE: tsan and asan cannot be used at the same time.
|
|
|
|
|
|
|
|
## Adddress sanitizer (asan)
|
|
|
|
|
|
|
|
To compile with the address sanitizer library (asan), you must set clang as your
|
2023-12-01 17:32:19 +00:00
|
|
|
compiler and set `-fsanitize=address` in compiler and linker flags.
|
build: Stop using hermetic clang, libc++, etc
This brings our default build config more in line with what is
necessary for some platforms anyway: using the system-installed
toolchain and sysroot to build everything.
We will no longer fetch source or binaries for any specific build
tools, such as libc++, clang, gold, binutils, or valgrind.
The main part of this change is the changing of default gyp settings
in gyp_packager.py. For this, a bug in gyp_packager.py had to be
fixed, in which similar GYP_DEFINE key names (such as clang and
host_clang) would conflict, causing some defaults not to be installed
properly.
In order to enable clang=0 by default, some changes had to be made in
common.gypi:
- compiler macros added to fix a compatibility issue between
Chromium's base/mac/ folder and the actual OSX SDK
- replaced clang_warning_flags variables with standard cflags
settings, plus xcode_settings for OSX
- turned off warnings-as-errors for non-shaka code, rather than
allow-listing specific warning types, since we can't actually fix
those warnings on any platform
- disabled two specific warnings in shaka code, both of which are
caused by headers from our non-shaka dependencies
Also, one warning (missing "override" keyword) has been fixed in
vod_media_info_dump_muxer_listener.h.
Although these changes were done to make building simpler on a wider
array of platforms (arm64, for example), it seems to make the build a
bit faster, too. For me, at least, on my main Linux workstation:
- "gclient sync" now runs 20-30% faster
- "ninja -C out/Release" now runs 5-13% faster
The following environment variables are no longer required:
- DEPOT_TOOLS_WIN_TOOLCHAIN
- MACOSX_DEPLOYMENT_TARGET
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following GYP_DEFINES are no longer required for anyone:
- clang=0
- host_clang=0
- clang_xcode=1
- use_allocator=none
- use_experimental_allocator_shim=0
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following repos are no longer dependencies in gclient:
- binutils
- clang
- gold
- libc++
- libc++abi
- valgrind
The following gclient hooks have been removed:
- clang
- mac_toolchain
- sysroot
Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
|
|
|
|
|
|
|
NOTE: tsan and asan cannot be used at the same time.
|
|
|
|
|
|
|
|
## Leak sanitizer (lsan)
|
|
|
|
|
|
|
|
To compile with the leak sanitizer library (lsan), you must set clang as your
|
2023-12-01 17:32:19 +00:00
|
|
|
compiler and use `-fsanitize=leak` in compiler and linker flags.
|