[dvsim] Implement LsfLauncher

This is a first cut implementation of the LsfLauncher. There are several
items left as TODOs - they will be addressed later.

This implementation dispatches all targets (builds, runs, cov etc) as
job arrays by default. Builds are run discretely (array of 1 job) since
we consider each build to have specific job requirements that cannot be
shared with other builds (cpu/mem/disk/stack usage settings - these will
be added in future). Runs pertaining to a build is dispatched as an
array. The associated changes made to other sources support the array
generation.

The job polling is not done by invoking bjobs or bhist, but by looking
for the LSF job output file (unique for each array index), which gets
written to only AFTER the job is complete. This offers a really fast way
to test for completion rather than invoking bjobs or bhist, which bring
the system to a crawl when invoked for 20k tests in flight. This largely
works for now, but we need to explore other options such as using IBM's
Platform LSF Python APIs (future work!).

What launcher system to pick is decided by `DVSIM_LAUNCHER` variable.
In addition, this PR also adds support for Python virtualenv to isolate
project-specific python requirements that need to be met when running
tasks on remote machines used by several other projects as well.

Signed-off-by: Srikrishna Iyer <sriyer@google.com>
7 files changed
tree: 6d8de651eedb6199321de38cd928f8f8d1718b65
  1. .github/
  2. ci/
  3. doc/
  4. hw/
  5. site/
  6. sw/
  7. test/
  8. util/
  9. .clang-format
  10. .dockerignore
  11. .flake8
  12. .gitignore
  13. .style.yapf
  14. .svlint.toml
  15. .svls.toml
  16. _index.md
  17. apt-requirements.txt
  18. azure-pipelines.yml
  19. check_tool_requirements.core
  20. CLA
  21. COMMITTERS
  22. CONTRIBUTING.md
  23. LICENSE
  24. meson.build
  25. meson_init.sh
  26. meson_options.txt
  27. python-requirements.txt
  28. README.md
  29. tool_requirements.py
  30. toolchain.txt
  31. topgen-generator.core
  32. topgen-reg-only.core
  33. topgen.core
  34. yum-requirements.txt
README.md

OpenTitan

OpenTitan logo

About the project

OpenTitan is an open source silicon Root of Trust (RoT) project. OpenTitan will make the silicon RoT design and implementation more transparent, trustworthy, and secure for enterprises, platform providers, and chip manufacturers. OpenTitan is administered by lowRISC CIC as a collaborative project to produce high quality, open IP for instantiation as a full-featured product. See the OpenTitan site and OpenTitan docs for more information about the project.

About this repository

This repository contains hardware, software and utilities written as part of the OpenTitan project. It is structured as monolithic repository, or “monorepo”, where all components live in one repository. It exists to enable collaboration across partners participating in the OpenTitan project.

Documentation

The project contains comprehensive documentation of all IPs and tools. You can access it online at docs.opentitan.org.

How to contribute

Have a look at CONTRIBUTING for guidelines on how to contribute code to this repository.

Licensing

Unless otherwise noted, everything in this repository is covered by the Apache License, Version 2.0 (see LICENSE for full text).