shaka-packager/packager/file/thread_pool.h

56 lines
1.5 KiB
C
Raw Normal View History

// Copyright 2022 Google LLC. All rights reserved.
feat: First phase of CMake build system implementation (#1072) There are a lot of changes in this first phase, because there was a lot of infrastructure required to get some meaningful amount of porting done. Future PRs should be simpler. <b>Summary of changes:</b><details> - Remove old deps: - boringssl (replaced with mbedtls, lighter, easier to build) - gflags (replaced with absl::flags) - Chromium build tools - New deps to replace parts of Chromium base: - abseil-cpp - glog - nlohmann::json (for tests only) - Submodules, updates, and CMake build rules for third-party libraries: - curl - gmock/gtest - Ported internal libraries and their tests by removing Chromium deps and adding CMake build rules: - file (now using C++17 filesystem APIs) - license_notice - status - version - Test improvements - Removed file tests that can never be re-enabled - Re-enabled all other disabled file tests - Debug JSON values when HTTP tests fail - Fixed chunked-encoding issues in HTTP tests - Updated and refactored Dockerfiles testing - All docker files working, with OS versions updated to meet the new tool requirements - Local docker builds no longer write files to your working directory as root - Local docker builds can now be run in parallel without clobbering each others' build outputs - DEBUG=1 can drop you into an interactive shell when a docker build fails - Updated and heavily refactored workflows and Dockerfiles - All docker files now tested in parallel on GitHub, speeding up CI - All common workflow components broken out and using workflow_call instead of custom actions - Self-hosted runners now optional, to make testing easier on forks - CMake porting works-in-process can now be fully tested on GitHub - Building ported libraries and passing ported tests on all three platforms! - CI hacks for macOS removed, now testing on macos-latest! - Python2 no longer required! (Only Python3) - Using strict build flags, treating all warnings as errors. </details> <b>Required to build:</b> - CMake >= 3.16 - Python 3 - A compiler supporting C++ >= 17 - g++ >= 9 if using GCC (Clang also fine) - MSVC for Windows <b>Still needs work:</b><details> - Moving other dependencies into submodules (if we keep them): - apple_apsl - icu - libevent - libpng - libwebm - libxml - modp_b64 - protobuf - zlib - Port remaining internal libraries: - app - hls - media/base - media/chunking - media/codecs - media/crypto - media/demuxer - media/event - media/formats/dvb - media/formats/mp2t - media/formats/mp4 - media/formats/packed_audio - media/formats/ttml - media/formats/webm - media/formats/webvtt - media/formats/wvm - media/origin - media/public - media/replicator - media/trick_play - mpd - Port main application - Add logging flags in absl and connect them to glog (which expects gflags) - Port pssh-box.py - Port main test targets (packager_test.py and packager_app.py) - Updating all requirement and build documentation - Remove any remaining refs to gclient, depot_tools, ninja - Update and complete release workflows using release-please </details> Issue #346 (Switch to abseil) Issue #1047 (New build system)
2022-08-16 18:34:51 +00:00
//
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file or at
// https://developers.google.com/open-source/licenses/bsd
#ifndef PACKAGER_FILE_THREAD_POOL_H_
#define PACKAGER_FILE_THREAD_POOL_H_
#include <functional>
#include <queue>
#include <absl/base/thread_annotations.h>
#include <absl/synchronization/mutex.h>
#include "packager/macros.h"
feat: First phase of CMake build system implementation (#1072) There are a lot of changes in this first phase, because there was a lot of infrastructure required to get some meaningful amount of porting done. Future PRs should be simpler. <b>Summary of changes:</b><details> - Remove old deps: - boringssl (replaced with mbedtls, lighter, easier to build) - gflags (replaced with absl::flags) - Chromium build tools - New deps to replace parts of Chromium base: - abseil-cpp - glog - nlohmann::json (for tests only) - Submodules, updates, and CMake build rules for third-party libraries: - curl - gmock/gtest - Ported internal libraries and their tests by removing Chromium deps and adding CMake build rules: - file (now using C++17 filesystem APIs) - license_notice - status - version - Test improvements - Removed file tests that can never be re-enabled - Re-enabled all other disabled file tests - Debug JSON values when HTTP tests fail - Fixed chunked-encoding issues in HTTP tests - Updated and refactored Dockerfiles testing - All docker files working, with OS versions updated to meet the new tool requirements - Local docker builds no longer write files to your working directory as root - Local docker builds can now be run in parallel without clobbering each others' build outputs - DEBUG=1 can drop you into an interactive shell when a docker build fails - Updated and heavily refactored workflows and Dockerfiles - All docker files now tested in parallel on GitHub, speeding up CI - All common workflow components broken out and using workflow_call instead of custom actions - Self-hosted runners now optional, to make testing easier on forks - CMake porting works-in-process can now be fully tested on GitHub - Building ported libraries and passing ported tests on all three platforms! - CI hacks for macOS removed, now testing on macos-latest! - Python2 no longer required! (Only Python3) - Using strict build flags, treating all warnings as errors. </details> <b>Required to build:</b> - CMake >= 3.16 - Python 3 - A compiler supporting C++ >= 17 - g++ >= 9 if using GCC (Clang also fine) - MSVC for Windows <b>Still needs work:</b><details> - Moving other dependencies into submodules (if we keep them): - apple_apsl - icu - libevent - libpng - libwebm - libxml - modp_b64 - protobuf - zlib - Port remaining internal libraries: - app - hls - media/base - media/chunking - media/codecs - media/crypto - media/demuxer - media/event - media/formats/dvb - media/formats/mp2t - media/formats/mp4 - media/formats/packed_audio - media/formats/ttml - media/formats/webm - media/formats/webvtt - media/formats/wvm - media/origin - media/public - media/replicator - media/trick_play - mpd - Port main application - Add logging flags in absl and connect them to glog (which expects gflags) - Port pssh-box.py - Port main test targets (packager_test.py and packager_app.py) - Updating all requirement and build documentation - Remove any remaining refs to gclient, depot_tools, ninja - Update and complete release workflows using release-please </details> Issue #346 (Switch to abseil) Issue #1047 (New build system)
2022-08-16 18:34:51 +00:00
namespace shaka {
/// A simple thread pool. We used to get this from Chromium base::, but there
/// is no replacement in the C++ standard library nor in absl.
/// (As of June 2022.) The pool will grow when there are no threads available
/// to handle a task, and it will shrink when a thread is idle for too long.
class ThreadPool {
public:
typedef std::function<void()> Task;
ThreadPool();
~ThreadPool();
/// Find or spawn a worker thread to handle |task|.
/// @param task A potentially long-running task to be handled by the pool.
void PostTask(const Task& task);
static ThreadPool instance;
private:
/// Stop handing out tasks to workers, wake up all threads, and make them
/// exit.
void Terminate();
Task WaitForTask();
void ThreadMain();
absl::Mutex mutex_;
absl::CondVar tasks_available_ ABSL_GUARDED_BY(mutex_);
std::queue<Task> tasks_ ABSL_GUARDED_BY(mutex_);
size_t num_idle_threads_ ABSL_GUARDED_BY(mutex_);
bool terminated_ ABSL_GUARDED_BY(mutex_);
feat: First phase of CMake build system implementation (#1072) There are a lot of changes in this first phase, because there was a lot of infrastructure required to get some meaningful amount of porting done. Future PRs should be simpler. <b>Summary of changes:</b><details> - Remove old deps: - boringssl (replaced with mbedtls, lighter, easier to build) - gflags (replaced with absl::flags) - Chromium build tools - New deps to replace parts of Chromium base: - abseil-cpp - glog - nlohmann::json (for tests only) - Submodules, updates, and CMake build rules for third-party libraries: - curl - gmock/gtest - Ported internal libraries and their tests by removing Chromium deps and adding CMake build rules: - file (now using C++17 filesystem APIs) - license_notice - status - version - Test improvements - Removed file tests that can never be re-enabled - Re-enabled all other disabled file tests - Debug JSON values when HTTP tests fail - Fixed chunked-encoding issues in HTTP tests - Updated and refactored Dockerfiles testing - All docker files working, with OS versions updated to meet the new tool requirements - Local docker builds no longer write files to your working directory as root - Local docker builds can now be run in parallel without clobbering each others' build outputs - DEBUG=1 can drop you into an interactive shell when a docker build fails - Updated and heavily refactored workflows and Dockerfiles - All docker files now tested in parallel on GitHub, speeding up CI - All common workflow components broken out and using workflow_call instead of custom actions - Self-hosted runners now optional, to make testing easier on forks - CMake porting works-in-process can now be fully tested on GitHub - Building ported libraries and passing ported tests on all three platforms! - CI hacks for macOS removed, now testing on macos-latest! - Python2 no longer required! (Only Python3) - Using strict build flags, treating all warnings as errors. </details> <b>Required to build:</b> - CMake >= 3.16 - Python 3 - A compiler supporting C++ >= 17 - g++ >= 9 if using GCC (Clang also fine) - MSVC for Windows <b>Still needs work:</b><details> - Moving other dependencies into submodules (if we keep them): - apple_apsl - icu - libevent - libpng - libwebm - libxml - modp_b64 - protobuf - zlib - Port remaining internal libraries: - app - hls - media/base - media/chunking - media/codecs - media/crypto - media/demuxer - media/event - media/formats/dvb - media/formats/mp2t - media/formats/mp4 - media/formats/packed_audio - media/formats/ttml - media/formats/webm - media/formats/webvtt - media/formats/wvm - media/origin - media/public - media/replicator - media/trick_play - mpd - Port main application - Add logging flags in absl and connect them to glog (which expects gflags) - Port pssh-box.py - Port main test targets (packager_test.py and packager_app.py) - Updating all requirement and build documentation - Remove any remaining refs to gclient, depot_tools, ninja - Update and complete release workflows using release-please </details> Issue #346 (Switch to abseil) Issue #1047 (New build system)
2022-08-16 18:34:51 +00:00
DISALLOW_COPY_AND_ASSIGN(ThreadPool);
};
} // namespace shaka
#endif // PACKAGER_FILE_THREAD_POOL_H_