shaka-packager/docs/linux_profiling.md

84 lines
3.0 KiB
Markdown
Raw Permalink Normal View History

# Linux Profiling
In theory we should be able to build packager using
[gperftools](https://github.com/gperftools/gperftools/tree/master) to
get back the profiling functionality described below. However actually
integrating this into the CMake build is not yet done. Pull requests
welcome. See https://github.com/shaka-project/shaka-packager/issues/1277
If packager was linked using `-ltcmalloc` then the following
instructions should work:
## CPU Profiling
In order to enable cpu profiling, run shaka-packager with the environment
variable `CPUPROFILE` set to a filename. For example:
CPUPROFILE=/tmp/cpuprofile out/Release/packager
The cpu profile will be dumped periodically to the filename specified in the
CPUPROFILE environment variable. You can then analyze the dumps using the pprof
script (`packager/third_party/tcmalloc/chromium/src/pprof`). For example,
pprof --gv out/Release/packager /tmp/cpuprofile
This will generate a visual representation of the cpu profile as a postscript
file and load it up using `gv`. For more powerful commands, please refer to the
pprof help output and the google-perftools documentation.
For further information, please refer to
http://google-perftools.googlecode.com/svn/trunk/doc/cpuprofile.html.
## Heap Profiling
To turn on the heap profiler on shaka-packager, use the `HEAPPROFILE`
environment variable to specify a filename for the heap profile. For example:
HEAPPROFILE=/tmp/heapprofile out/Release/packager
The heap profile will be dumped periodically to the filename specified in the
`HEAPPROFILE` environment variable. The dumps can be analyzed using the same
command as cpu profiling above.
For further information, please refer to
http://google-perftools.googlecode.com/svn/trunk/doc/heapprofile.html.
Some tests fork short-living processes which have a small memory footprint. To
catch those, use the `HEAP_PROFILE_ALLOCATION_INTERVAL` environment variable.
#### Dumping a profile of a running process
To programmatically generate a heap profile before exit, use code like:
#include <gperftools/heap-profiler.h>
// "foobar" will be included in the message printed to the console
HeapProfilerDump("foobar");
Or you can use gdb to attach at any point:
1. Attach gdb to the process: `$ gdb -p 12345`
2. Cause it to dump a profile: `(gdb) p HeapProfilerDump("foobar")`
3. The filename will be printed on the console, e.g.
"`Dumping heap profile to heap.0001.heap (foobar)`"
build: Stop using hermetic clang, libc++, etc This brings our default build config more in line with what is necessary for some platforms anyway: using the system-installed toolchain and sysroot to build everything. We will no longer fetch source or binaries for any specific build tools, such as libc++, clang, gold, binutils, or valgrind. The main part of this change is the changing of default gyp settings in gyp_packager.py. For this, a bug in gyp_packager.py had to be fixed, in which similar GYP_DEFINE key names (such as clang and host_clang) would conflict, causing some defaults not to be installed properly. In order to enable clang=0 by default, some changes had to be made in common.gypi: - compiler macros added to fix a compatibility issue between Chromium's base/mac/ folder and the actual OSX SDK - replaced clang_warning_flags variables with standard cflags settings, plus xcode_settings for OSX - turned off warnings-as-errors for non-shaka code, rather than allow-listing specific warning types, since we can't actually fix those warnings on any platform - disabled two specific warnings in shaka code, both of which are caused by headers from our non-shaka dependencies Also, one warning (missing "override" keyword) has been fixed in vod_media_info_dump_muxer_listener.h. Although these changes were done to make building simpler on a wider array of platforms (arm64, for example), it seems to make the build a bit faster, too. For me, at least, on my main Linux workstation: - "gclient sync" now runs 20-30% faster - "ninja -C out/Release" now runs 5-13% faster The following environment variables are no longer required: - DEPOT_TOOLS_WIN_TOOLCHAIN - MACOSX_DEPLOYMENT_TARGET Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following GYP_DEFINES are no longer required for anyone: - clang=0 - host_clang=0 - clang_xcode=1 - use_allocator=none - use_experimental_allocator_shim=0 Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following repos are no longer dependencies in gclient: - binutils - clang - gold - libc++ - libc++abi - valgrind The following gclient hooks have been removed: - clang - mac_toolchain - sysroot Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
## Thread sanitizer (tsan)
To compile with the thread sanitizer library (tsan), you must set clang as your
compiler and set `-fsanitize=thread` in compiler flags.
build: Stop using hermetic clang, libc++, etc This brings our default build config more in line with what is necessary for some platforms anyway: using the system-installed toolchain and sysroot to build everything. We will no longer fetch source or binaries for any specific build tools, such as libc++, clang, gold, binutils, or valgrind. The main part of this change is the changing of default gyp settings in gyp_packager.py. For this, a bug in gyp_packager.py had to be fixed, in which similar GYP_DEFINE key names (such as clang and host_clang) would conflict, causing some defaults not to be installed properly. In order to enable clang=0 by default, some changes had to be made in common.gypi: - compiler macros added to fix a compatibility issue between Chromium's base/mac/ folder and the actual OSX SDK - replaced clang_warning_flags variables with standard cflags settings, plus xcode_settings for OSX - turned off warnings-as-errors for non-shaka code, rather than allow-listing specific warning types, since we can't actually fix those warnings on any platform - disabled two specific warnings in shaka code, both of which are caused by headers from our non-shaka dependencies Also, one warning (missing "override" keyword) has been fixed in vod_media_info_dump_muxer_listener.h. Although these changes were done to make building simpler on a wider array of platforms (arm64, for example), it seems to make the build a bit faster, too. For me, at least, on my main Linux workstation: - "gclient sync" now runs 20-30% faster - "ninja -C out/Release" now runs 5-13% faster The following environment variables are no longer required: - DEPOT_TOOLS_WIN_TOOLCHAIN - MACOSX_DEPLOYMENT_TARGET Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following GYP_DEFINES are no longer required for anyone: - clang=0 - host_clang=0 - clang_xcode=1 - use_allocator=none - use_experimental_allocator_shim=0 Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following repos are no longer dependencies in gclient: - binutils - clang - gold - libc++ - libc++abi - valgrind The following gclient hooks have been removed: - clang - mac_toolchain - sysroot Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
NOTE: tsan and asan cannot be used at the same time.
## Adddress sanitizer (asan)
To compile with the address sanitizer library (asan), you must set clang as your
compiler and set `-fsanitize=address` in compiler and linker flags.
build: Stop using hermetic clang, libc++, etc This brings our default build config more in line with what is necessary for some platforms anyway: using the system-installed toolchain and sysroot to build everything. We will no longer fetch source or binaries for any specific build tools, such as libc++, clang, gold, binutils, or valgrind. The main part of this change is the changing of default gyp settings in gyp_packager.py. For this, a bug in gyp_packager.py had to be fixed, in which similar GYP_DEFINE key names (such as clang and host_clang) would conflict, causing some defaults not to be installed properly. In order to enable clang=0 by default, some changes had to be made in common.gypi: - compiler macros added to fix a compatibility issue between Chromium's base/mac/ folder and the actual OSX SDK - replaced clang_warning_flags variables with standard cflags settings, plus xcode_settings for OSX - turned off warnings-as-errors for non-shaka code, rather than allow-listing specific warning types, since we can't actually fix those warnings on any platform - disabled two specific warnings in shaka code, both of which are caused by headers from our non-shaka dependencies Also, one warning (missing "override" keyword) has been fixed in vod_media_info_dump_muxer_listener.h. Although these changes were done to make building simpler on a wider array of platforms (arm64, for example), it seems to make the build a bit faster, too. For me, at least, on my main Linux workstation: - "gclient sync" now runs 20-30% faster - "ninja -C out/Release" now runs 5-13% faster The following environment variables are no longer required: - DEPOT_TOOLS_WIN_TOOLCHAIN - MACOSX_DEPLOYMENT_TARGET Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following GYP_DEFINES are no longer required for anyone: - clang=0 - host_clang=0 - clang_xcode=1 - use_allocator=none - use_experimental_allocator_shim=0 Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following repos are no longer dependencies in gclient: - binutils - clang - gold - libc++ - libc++abi - valgrind The following gclient hooks have been removed: - clang - mac_toolchain - sysroot Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
NOTE: tsan and asan cannot be used at the same time.
## Leak sanitizer (lsan)
To compile with the leak sanitizer library (lsan), you must set clang as your
compiler and use `-fsanitize=leak` in compiler and linker flags.