[dvsim] Implement LsfLauncher
This is a first cut implementation of the LsfLauncher. There are several
items left as TODOs - they will be addressed later.
This implementation dispatches all targets (builds, runs, cov etc) as
job arrays by default. Builds are run discretely (array of 1 job) since
we consider each build to have specific job requirements that cannot be
shared with other builds (cpu/mem/disk/stack usage settings - these will
be added in future). Runs pertaining to a build is dispatched as an
array. The associated changes made to other sources support the array
generation.
The job polling is not done by invoking bjobs or bhist, but by looking
for the LSF job output file (unique for each array index), which gets
written to only AFTER the job is complete. This offers a really fast way
to test for completion rather than invoking bjobs or bhist, which bring
the system to a crawl when invoked for 20k tests in flight. This largely
works for now, but we need to explore other options such as using IBM's
Platform LSF Python APIs (future work!).
What launcher system to pick is decided by `DVSIM_LAUNCHER` variable.
In addition, this PR also adds support for Python virtualenv to isolate
project-specific python requirements that need to be met when running
tasks on remote machines used by several other projects as well.
Signed-off-by: Srikrishna Iyer <sriyer@google.com>
diff --git a/util/dvsim/Deploy.py b/util/dvsim/Deploy.py
index b0114a8..e055b81 100644
--- a/util/dvsim/Deploy.py
+++ b/util/dvsim/Deploy.py
@@ -6,7 +6,7 @@
import pprint
import random
-from LocalLauncher import LocalLauncher
+from LauncherFactory import get_launcher
from sim_utils import get_cov_summary_table
from tabulate import tabulate
from utils import (VERBOSE, clean_odirs, find_and_substitute_wildcards,
@@ -64,7 +64,7 @@
# Create the launcher object. Launcher retains the handle to self for
# lookup & callbacks.
- self.launcher = LocalLauncher(self)
+ self.launcher = get_launcher(self)
def _define_attrs(self):
"""Defines the attributes this instance needs to have.
@@ -133,6 +133,9 @@
# 'aes:default', 'uart:default' builds.
self.full_name = self.sim_cfg.name + ":" + self.qual_name
+ # Job name is used to group the job by cfg and target.
+ self.job_name = "{}_{}".format(self.sim_cfg.name, self.target)
+
# Pass and fail patterns.
self.pass_patterns = []
self.fail_patterns = []
@@ -298,6 +301,8 @@
# 'build_mode' is used as a substitution variable in the HJson.
self.build_mode = self.name
+ self.job_name = "{}_{}_{}".format(self.sim_cfg.name, self.target,
+ self.build_mode)
self.pass_patterns = self.build_pass_patterns
self.fail_patterns = self.build_fail_patterns
@@ -346,6 +351,8 @@
# 'build_mode' is used as a substitution variable in the HJson.
self.build_mode = self.name
+ self.job_name = "{}_{}_{}".format(self.sim_cfg.name, self.target,
+ self.build_mode)
class RunTest(Deploy):
@@ -366,6 +373,10 @@
if build_job is not None:
self.dependencies.append(build_job)
+ # We did something wrong if build_mode is not the same as the build_job
+ # arg's name.
+ assert self.build_mode == build_job.name
+
self.launcher.renew_odir = True
def _define_attrs(self):
@@ -402,6 +413,8 @@
self.build_mode = self.test_obj.build_mode.name
self.qual_name = self.run_dir_name + "." + str(self.seed)
self.full_name = self.sim_cfg.name + ":" + self.qual_name
+ self.job_name = "{}_{}_{}".format(self.sim_cfg.name, self.target,
+ self.build_mode)
self.pass_patterns = self.run_pass_patterns
self.fail_patterns = self.run_fail_patterns
