Commit Graph

16 Commits

Author SHA1 Message Date
Joey Parrish 7653e1e02f
feat: Replace glog with absl::log (#1274)
This deprecates --vmodule (not available in absl, mapping it to --v for
a very minimal sort of compatibility) and adds the absl::log flags
--minloglevel, --stderrthreshold, --log_backtrace_at, and --log_prefix.
2023-10-13 12:42:47 -07:00
Joey Parrish 3b3e267221
style: Sort headers to comply with the style guide (#1273)
Reorder headers to follow the Google C++ Style Guide:

> In dir/foo.cc or dir/foo_test.cc:
>
> 1. dir2/foo2.h.
> 2. A blank line
> 3. C system headers (more precisely: headers in angle brackets with
the .h extension), e.g., <unistd.h>, <stdlib.h>.
> 4. A blank line
> 5. C++ standard library headers (without file extension), e.g.,
<algorithm>, <cstddef>.
> 6. A blank line
> 7. Other libraries' .h files.
> 8. A blank line
> 9. Your project's .h files.


https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes
2023-10-11 16:49:50 +08:00
Joey Parrish 4e310f03bf
style: Use angle-brackets for all internal headers (#1271)
This feeds into efforts to create a working install target.

The order of headers is still funky, and was "fixed" by clang-format,
but in a way that doesn't exactly align with the style guide. Further
cleanup of header order is coming in a follow-up PR.
2023-10-11 07:51:11 +08:00
Cosmin Stejerean 1337e86597
style: cleanup unused includes and use consistent <> for external code (#1267) 2023-10-09 16:21:41 -07:00
Joey Parrish af98d48726
fix: Fix flush/close semantics for HTTP files, improve testing (#1232)
All HTTP-based tests now use an embedded test server instead of
httpbin.org, which makes them much faster and more reliable.

These more reliable tests also exposed some issues that began recently
with PR #1201.  HttpFile's Flush() semantics were different than those
documented for files in general.  Flush() used to close the file for
uploading, so that no further writes were allowed, but the documentation
stated that it would only flush data to its destination.  PR #1201
brought HttpFile's Flush() in line with the docs, but gave us no way to
terminate a chunked upload.

This adds a new method to File called CloseForWriting(), which
terminates a chunked upload for HttpFile.  The only other implementation
that does anything is UdpFile, which uses the socket library function
shutdown() to terminate writes while allowing reads.

This also tweaks HttpFile::CloseWithStatus() so that it will not
generate an error if the file is closed before the HTTP response is
written to the download cache.

This modifies the test HttpFileTest.MultipleWrites so that the file is
Flushed after each chunk.  This adds test coverage for the changes
introduced in PR #1201.

Fixes #1224 (missing test coverage for HttpFile::Flush)
2023-07-13 18:55:48 -07:00
Joey Parrish 1131bf7eea
chore: Mass-update copyright headers (#1086)
Closes #1076
2022-08-26 08:44:59 -07:00
Joey Parrish e6b57f72a8
feat: First phase of CMake build system implementation (#1072)
There are a lot of changes in this first phase, because there was a
lot of infrastructure required to get some meaningful amount of
porting done.  Future PRs should be simpler.

<b>Summary of changes:</b><details>

 - Remove old deps:
   - boringssl (replaced with mbedtls, lighter, easier to build)
   - gflags (replaced with absl::flags)
   - Chromium build tools
 - New deps to replace parts of Chromium base:
   - abseil-cpp
   - glog
   - nlohmann::json (for tests only)
 - Submodules, updates, and CMake build rules for third-party
   libraries:
   - curl
   - gmock/gtest
 - Ported internal libraries and their tests by removing Chromium deps
   and adding CMake build rules:
   - file (now using C++17 filesystem APIs)
   - license_notice
   - status
   - version
 - Test improvements
   - Removed file tests that can never be re-enabled
   - Re-enabled all other disabled file tests
   - Debug JSON values when HTTP tests fail
   - Fixed chunked-encoding issues in HTTP tests
 - Updated and refactored Dockerfiles testing
   - All docker files working, with OS versions updated to meet the
     new tool requirements
   - Local docker builds no longer write files to your working
     directory as root
   - Local docker builds can now be run in parallel without clobbering
     each others' build outputs
   - DEBUG=1 can drop you into an interactive shell when a docker
     build fails
 - Updated and heavily refactored workflows and Dockerfiles
   - All docker files now tested in parallel on GitHub, speeding up CI
   - All common workflow components broken out and using workflow_call
     instead of custom actions
   - Self-hosted runners now optional, to make testing easier on forks
   - CMake porting works-in-process can now be fully tested on GitHub
   - Building ported libraries and passing ported tests on all three
     platforms!
 - CI hacks for macOS removed, now testing on macos-latest!
 - Python2 no longer required!  (Only Python3)
 - Using strict build flags, treating all warnings as errors.

</details>

<b>Required to build:</b>

 - CMake >= 3.16
 - Python 3
 - A compiler supporting C++ >= 17
   - g++ >= 9 if using GCC (Clang also fine)
   - MSVC for Windows

<b>Still needs work:</b><details>

 - Moving other dependencies into submodules (if we keep them):
   - apple_apsl
   - icu
   - libevent
   - libpng
   - libwebm
   - libxml
   - modp_b64
   - protobuf
   - zlib
 - Port remaining internal libraries:
   - app
   - hls
   - media/base
   - media/chunking
   - media/codecs
   - media/crypto
   - media/demuxer
   - media/event
   - media/formats/dvb
   - media/formats/mp2t
   - media/formats/mp4
   - media/formats/packed_audio
   - media/formats/ttml
   - media/formats/webm
   - media/formats/webvtt
   - media/formats/wvm
   - media/origin
   - media/public
   - media/replicator
   - media/trick_play
   - mpd
 - Port main application
   - Add logging flags in absl and connect them to glog (which expects
     gflags)
 - Port pssh-box.py
 - Port main test targets (packager_test.py and packager_app.py)
 - Updating all requirement and build documentation
 - Remove any remaining refs to gclient, depot_tools, ninja
 - Update and complete release workflows using release-please
</details>

Issue #346 (Switch to abseil)
Issue #1047 (New build system)
2022-08-16 11:34:51 -07:00
KongQun Yang ff5f3f3abc Call WSAStartup once for every new socket
Some users are seeing problems with only one WSAStartup call.

WSAStartup can be called more than once:
https://docs.microsoft.com/en-us/windows/win32/api/winsock/nf-winsock-wsastartup

Fixes #643.

Change-Id: I81e0e7979d6a586da452984a96a8557b7b3ce7f6
2019-11-20 14:00:51 -08:00
KongQun Yang 3ec2975e36 Log socket error code on failure
Issue #643.

Change-Id: I8adfb9910e1e43294738abb7783a64645a4f42c3
2019-09-10 21:44:28 +00:00
Tomohiro IKEDA f810fea0ef Cleanup condition on `bind` error (#624) 2019-08-01 17:32:28 -07:00
KongQun Yang 715ed939f1 Add instructions to build on Alpine Linux
- Also fixed compilations in Alpine Linux and other flavors of Linux.
- Added container versions in docker files to always use a verified
  version.

Closes #164.

Change-Id: I949a8709e4d70c49129c9c2e8608dd78193d964c
2018-08-17 20:46:11 +00:00
KongQun Yang d3903fad19 Add buffer_size UDP file options
This allows UDP receive buffer size to be increased to avoid potential
packet loss result from receive buffer overrun.

Another related flag is --io_cache_size. buffer_size in UDP options
defines the UDP buffer size of the underlying system while
io_cache_size defines the size of the internal circular buffer managed
by Shaka Packager.

Closes #411

Change-Id: I57c843a88f13da546417dadc2a78df2bee0a00f3
2018-06-29 00:19:30 +00:00
fatpelt 6a8d2aa443 Add Source Specific Multicast (SSM) support
SSM can be enabled using "source" option in UDP options. Previously, "source" is considered the same as "interface", which is incorrect. 

Fixes #332.
2018-03-06 10:09:28 -08:00
KongQun Yang 82e0fd2095 Default multicast interface address to "0.0.0.0" if absent
It aligns with what a lot of other apps are doing, e.g. tsplay.

Also added notes in docker to adjust network setting if wants to use
multicast in docker container.

Change-Id: I666a8979cc041c904f7ff8e3e022f800c0830d5d
2017-12-04 19:21:52 +00:00
KongQun Yang 723079310f Disable IP_MULTICAST_ALL when setting up UDP multicast
This avoids interference caused when two sockets are bound to the
same port in different multicast groups.

Fixes #241

Change-Id: Iba1b3300e3850f1f268886b7a49f5a3315d95b80
2017-07-27 18:02:51 -07:00
KongQun Yang eb98a87f7d Move packager/media/file to packager/file
Change-Id: Idc0d75a7ceab7c8ce460ef896fdefdec088124bb
2017-07-17 12:08:28 -07:00