shaka-packager/gyp_packager.py

125 lines
4.4 KiB
Python
Raw Normal View History

#!/usr/bin/python3
#
# Copyright 2014 Google Inc. All rights reserved.
#
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file or at
# https://developers.google.com/open-source/licenses/bsd
"""This script wraps gyp and sets up build environments.
Build instructions:
1. Setup gyp: ./gyp_packager.py or use gclient runhooks
Ninja is the default build system. User can also change to make by
overriding GYP_GENERATORS to make, i.e.
"GYP_GENERATORS='make' gclient runhooks".
2. The first step generates the make files but does not start the
build process. Ninja is the default build system. Refer to Ninja
manual on how to do the build.
Common syntaxes: ninja -C out/{Debug/Release} [Module]
Module is optional. If not specified, build everything.
Step 1 is only required if there is any gyp file change. Otherwise, you
may just run ninja.
"""
import os
import sys
checkout_dir = os.path.dirname(os.path.realpath(__file__))
src_dir = os.path.join(checkout_dir, 'packager')
# Workaround the dynamic path.
# pylint: disable=wrong-import-position
sys.path.insert(0, os.path.join(src_dir, 'build'))
import gyp_helper
sys.path.insert(0, os.path.join(src_dir, 'tools', 'gyp', 'pylib'))
import gyp
if __name__ == '__main__':
args = sys.argv[1:]
# Allow src/.../chromium.gyp_env to define GYP variables.
gyp_helper.apply_chromium_gyp_env()
# If we didn't get a gyp file, then fall back to assuming 'packager.gyp' from
# the same directory as the script.
if not any(arg.endswith('.gyp') for arg in args):
args.append(os.path.join(src_dir, 'packager.gyp'))
# Always include Chromium's common.gypi and our common.gypi.
args.extend([
'-I' + os.path.join(src_dir, 'build', 'common.gypi'),
'-I' + os.path.join(src_dir, 'common.gypi')
])
# Set these default GYP_DEFINES if user does not set the value explicitly.
_DEFAULT_DEFINES = {'test_isolation_mode': 'noop',
'use_custom_libcxx': 0,
'use_glib': 0,
'use_openssl': 1,
'use_sysroot': 0,
'use_x11': 0,
'linux_use_bundled_binutils': 0,
'linux_use_bundled_gold': 0,
'linux_use_gold_flags': 0,
build: Stop using hermetic clang, libc++, etc This brings our default build config more in line with what is necessary for some platforms anyway: using the system-installed toolchain and sysroot to build everything. We will no longer fetch source or binaries for any specific build tools, such as libc++, clang, gold, binutils, or valgrind. The main part of this change is the changing of default gyp settings in gyp_packager.py. For this, a bug in gyp_packager.py had to be fixed, in which similar GYP_DEFINE key names (such as clang and host_clang) would conflict, causing some defaults not to be installed properly. In order to enable clang=0 by default, some changes had to be made in common.gypi: - compiler macros added to fix a compatibility issue between Chromium's base/mac/ folder and the actual OSX SDK - replaced clang_warning_flags variables with standard cflags settings, plus xcode_settings for OSX - turned off warnings-as-errors for non-shaka code, rather than allow-listing specific warning types, since we can't actually fix those warnings on any platform - disabled two specific warnings in shaka code, both of which are caused by headers from our non-shaka dependencies Also, one warning (missing "override" keyword) has been fixed in vod_media_info_dump_muxer_listener.h. Although these changes were done to make building simpler on a wider array of platforms (arm64, for example), it seems to make the build a bit faster, too. For me, at least, on my main Linux workstation: - "gclient sync" now runs 20-30% faster - "ninja -C out/Release" now runs 5-13% faster The following environment variables are no longer required: - DEPOT_TOOLS_WIN_TOOLCHAIN - MACOSX_DEPLOYMENT_TARGET Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following GYP_DEFINES are no longer required for anyone: - clang=0 - host_clang=0 - clang_xcode=1 - use_allocator=none - use_experimental_allocator_shim=0 Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following repos are no longer dependencies in gclient: - binutils - clang - gold - libc++ - libc++abi - valgrind The following gclient hooks have been removed: - clang - mac_toolchain - sysroot Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
'clang': 0,
'host_clang': 0,
'clang_xcode': 1,
'use_allocator': 'none',
'mac_deployment_target': '10.10',
'use_experimental_allocator_shim': 0,
'clang_use_chrome_plugins': 0}
build: Stop using hermetic clang, libc++, etc This brings our default build config more in line with what is necessary for some platforms anyway: using the system-installed toolchain and sysroot to build everything. We will no longer fetch source or binaries for any specific build tools, such as libc++, clang, gold, binutils, or valgrind. The main part of this change is the changing of default gyp settings in gyp_packager.py. For this, a bug in gyp_packager.py had to be fixed, in which similar GYP_DEFINE key names (such as clang and host_clang) would conflict, causing some defaults not to be installed properly. In order to enable clang=0 by default, some changes had to be made in common.gypi: - compiler macros added to fix a compatibility issue between Chromium's base/mac/ folder and the actual OSX SDK - replaced clang_warning_flags variables with standard cflags settings, plus xcode_settings for OSX - turned off warnings-as-errors for non-shaka code, rather than allow-listing specific warning types, since we can't actually fix those warnings on any platform - disabled two specific warnings in shaka code, both of which are caused by headers from our non-shaka dependencies Also, one warning (missing "override" keyword) has been fixed in vod_media_info_dump_muxer_listener.h. Although these changes were done to make building simpler on a wider array of platforms (arm64, for example), it seems to make the build a bit faster, too. For me, at least, on my main Linux workstation: - "gclient sync" now runs 20-30% faster - "ninja -C out/Release" now runs 5-13% faster The following environment variables are no longer required: - DEPOT_TOOLS_WIN_TOOLCHAIN - MACOSX_DEPLOYMENT_TARGET Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following GYP_DEFINES are no longer required for anyone: - clang=0 - host_clang=0 - clang_xcode=1 - use_allocator=none - use_experimental_allocator_shim=0 Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following repos are no longer dependencies in gclient: - binutils - clang - gold - libc++ - libc++abi - valgrind The following gclient hooks have been removed: - clang - mac_toolchain - sysroot Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
gyp_defines_str = os.environ.get('GYP_DEFINES', '')
user_gyp_defines_map = {}
for term in gyp_defines_str.split(' '):
if term:
key, value = term.strip().split('=')
user_gyp_defines_map[key] = value
for key, value in _DEFAULT_DEFINES.items():
build: Stop using hermetic clang, libc++, etc This brings our default build config more in line with what is necessary for some platforms anyway: using the system-installed toolchain and sysroot to build everything. We will no longer fetch source or binaries for any specific build tools, such as libc++, clang, gold, binutils, or valgrind. The main part of this change is the changing of default gyp settings in gyp_packager.py. For this, a bug in gyp_packager.py had to be fixed, in which similar GYP_DEFINE key names (such as clang and host_clang) would conflict, causing some defaults not to be installed properly. In order to enable clang=0 by default, some changes had to be made in common.gypi: - compiler macros added to fix a compatibility issue between Chromium's base/mac/ folder and the actual OSX SDK - replaced clang_warning_flags variables with standard cflags settings, plus xcode_settings for OSX - turned off warnings-as-errors for non-shaka code, rather than allow-listing specific warning types, since we can't actually fix those warnings on any platform - disabled two specific warnings in shaka code, both of which are caused by headers from our non-shaka dependencies Also, one warning (missing "override" keyword) has been fixed in vod_media_info_dump_muxer_listener.h. Although these changes were done to make building simpler on a wider array of platforms (arm64, for example), it seems to make the build a bit faster, too. For me, at least, on my main Linux workstation: - "gclient sync" now runs 20-30% faster - "ninja -C out/Release" now runs 5-13% faster The following environment variables are no longer required: - DEPOT_TOOLS_WIN_TOOLCHAIN - MACOSX_DEPLOYMENT_TARGET Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following GYP_DEFINES are no longer required for anyone: - clang=0 - host_clang=0 - clang_xcode=1 - use_allocator=none - use_experimental_allocator_shim=0 Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following repos are no longer dependencies in gclient: - binutils - clang - gold - libc++ - libc++abi - valgrind The following gclient hooks have been removed: - clang - mac_toolchain - sysroot Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
if key not in user_gyp_defines_map:
gyp_defines_str += ' {0}={1}'.format(key, value)
os.environ['GYP_DEFINES'] = gyp_defines_str.strip()
# Default to ninja, but only if no generator has explicitly been set.
if 'GYP_GENERATORS' not in os.environ:
os.environ['GYP_GENERATORS'] = 'ninja'
build: Stop using hermetic clang, libc++, etc This brings our default build config more in line with what is necessary for some platforms anyway: using the system-installed toolchain and sysroot to build everything. We will no longer fetch source or binaries for any specific build tools, such as libc++, clang, gold, binutils, or valgrind. The main part of this change is the changing of default gyp settings in gyp_packager.py. For this, a bug in gyp_packager.py had to be fixed, in which similar GYP_DEFINE key names (such as clang and host_clang) would conflict, causing some defaults not to be installed properly. In order to enable clang=0 by default, some changes had to be made in common.gypi: - compiler macros added to fix a compatibility issue between Chromium's base/mac/ folder and the actual OSX SDK - replaced clang_warning_flags variables with standard cflags settings, plus xcode_settings for OSX - turned off warnings-as-errors for non-shaka code, rather than allow-listing specific warning types, since we can't actually fix those warnings on any platform - disabled two specific warnings in shaka code, both of which are caused by headers from our non-shaka dependencies Also, one warning (missing "override" keyword) has been fixed in vod_media_info_dump_muxer_listener.h. Although these changes were done to make building simpler on a wider array of platforms (arm64, for example), it seems to make the build a bit faster, too. For me, at least, on my main Linux workstation: - "gclient sync" now runs 20-30% faster - "ninja -C out/Release" now runs 5-13% faster The following environment variables are no longer required: - DEPOT_TOOLS_WIN_TOOLCHAIN - MACOSX_DEPLOYMENT_TARGET Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following GYP_DEFINES are no longer required for anyone: - clang=0 - host_clang=0 - clang_xcode=1 - use_allocator=none - use_experimental_allocator_shim=0 Documentation, Dockerfiles, and GitHub Actions workflows have been updated to reflect this. The following repos are no longer dependencies in gclient: - binutils - clang - gold - libc++ - libc++abi - valgrind The following gclient hooks have been removed: - clang - mac_toolchain - sysroot Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
2021-07-28 20:01:19 +00:00
# By default, don't download our own toolchain for Windows.
if 'DEPOT_TOOLS_WIN_TOOLCHAIN' not in os.environ:
os.environ['DEPOT_TOOLS_WIN_TOOLCHAIN'] = '0'
# There shouldn't be a circular dependency relationship between .gyp files,
# but in Chromium's .gyp files, on non-Mac platforms, circular relationships
# currently exist. The check for circular dependencies is currently
# bypassed on other platforms, but is left enabled on the Mac, where a
# violation of the rule causes Xcode to misbehave badly.
if 'xcode' not in os.environ['GYP_GENERATORS']:
args.append('--no-circular-check')
# TODO(kqyang): Find a better way to handle the depth. This workaround works
# only if this script is executed in 'src' directory.
if not any('--depth' in arg for arg in args):
args.append('--depth=packager')
if 'output_dir=' not in os.environ.get('GYP_GENERATOR_FLAGS', ''):
output_dir = os.path.join(checkout_dir, 'out')
gyp_generator_flags = 'output_dir="' + output_dir + '"'
if os.environ.get('GYP_GENERATOR_FLAGS'):
os.environ['GYP_GENERATOR_FLAGS'] += ' ' + gyp_generator_flags
else:
os.environ['GYP_GENERATOR_FLAGS'] = gyp_generator_flags
print('Updating projects from gyp files...')
sys.stdout.flush()
# Off we go...
sys.exit(gyp.main(args))