90 lines
2.8 KiB
INI
90 lines
2.8 KiB
INI
|
# Copyright (c) 2013 The Chromium Authors. All rights reserved.
|
||
|
|
# Use of this source code is governed by a BSD-style license that can be
|
||
|
|
# found in the LICENSE file.
|
||
|
|
|
||
|
|
"""Config file for Run Performance Test Bisect Tool
|
||
|
|
|
||
|
|
This script is intended for use by anyone that wants to run a remote bisection
|
||
|
|
on a range of revisions to look for a performance regression. Modify the config
|
||
|
|
below and add the revision range, performance command, and metric. You can then
|
||
|
|
run a git try <bot>.
|
||
|
|
|
||
|
|
Changes to this file should never be submitted.
|
||
|
|
|
||
|
|
Args:
|
||
|
|
'command': This is the full command line to pass to the
|
||
|
|
bisect-perf-regression.py script in order to execute the test.
|
||
|
|
'good_revision': An svn or git revision where the metric hadn't regressed yet.
|
||
|
|
'bad_revision': An svn or git revision sometime after the metric had
|
||
|
|
regressed.
|
||
|
|
'metric': The name of the metric to parse out from the results of the
|
||
|
|
performance test. You can retrieve the metric by looking at the stdio of
|
||
|
|
the performance test. Look for lines of the format:
|
||
|
|
|
||
|
|
RESULT <graph>: <trace>= <value> <units>
|
||
|
|
|
||
|
|
The metric name is "<graph>/<trace>".
|
||
|
|
'repeat_count': The number of times to repeat the performance test.
|
||
|
|
'max_time_minutes': The script will attempt to run the performance test
|
||
|
|
"repeat_count" times, unless it exceeds "max_time_minutes".
|
||
|
|
'truncate_percent': Discard the highest/lowest % values from performance test.
|
||
|
|
|
||
|
|
Sample config:
|
||
|
|
|
||
|
|
config = {
|
||
|
|
'command': './out/Release/performance_ui_tests' +
|
||
|
|
' --gtest_filter=PageCyclerTest.Intl1File',
|
||
|
|
'good_revision': '179755',
|
||
|
|
'bad_revision': '179782',
|
||
|
|
'metric': 'times/t',
|
||
|
|
'repeat_count': '20',
|
||
|
|
'max_time_minutes': '20',
|
||
|
|
'truncate_percent': '25',
|
||
|
|
}
|
||
|
|
|
||
|
|
On Windows:
|
||
|
|
- If you're calling a python script you will need to add "python" to
|
||
|
|
the command:
|
||
|
|
|
||
|
|
config = {
|
||
|
|
'command': 'python tools/perf/run_measurement -v --browser=release kraken',
|
||
|
|
'good_revision': '185319',
|
||
|
|
'bad_revision': '185364',
|
||
|
|
'metric': 'Total/Total',
|
||
|
|
'repeat_count': '20',
|
||
|
|
'max_time_minutes': '20',
|
||
|
|
'truncate_percent': '25',
|
||
|
|
}
|
||
|
|
|
||
|
|
|
||
|
|
On ChromeOS:
|
||
|
|
- Script accepts either ChromeOS versions, or unix timestamps as revisions.
|
||
|
|
- You don't need to specify --identity and --remote, they will be added to
|
||
|
|
the command using the bot's BISECT_CROS_IP and BISECT_CROS_BOARD values.
|
||
|
|
|
||
|
|
config = {
|
||
|
|
'command': './tools/perf/run_measurement -v '\
|
||
|
|
'--browser=cros-chrome-guest '\
|
||
|
|
'dromaeo tools/perf/page_sets/dromaeo/jslibstylejquery.json',
|
||
|
|
'good_revision': '4086.0.0',
|
||
|
|
'bad_revision': '4087.0.0',
|
||
|
|
'metric': 'jslib/jslib',
|
||
|
|
'repeat_count': '20',
|
||
|
|
'max_time_minutes': '20',
|
||
|
|
'truncate_percent': '25',
|
||
|
|
}
|
||
|
|
|
||
|
|
"""
|
||
|
|
|
||
|
|
config = {
|
||
|
|
'command': '',
|
||
|
|
'good_revision': '',
|
||
|
|
'bad_revision': '',
|
||
|
|
'metric': '',
|
||
|
|
'repeat_count':'',
|
||
|
|
'max_time_minutes': '',
|
||
|
|
'truncate_percent':'',
|
||
|
|
}
|
||
|
|
|
||
|
|
# Workaround git try issue, see crbug.com/257689
|