[dvsim] Document and slightly improve subst_wildcards in utils.py
When messing around with config files, I managed to trigger an
infinite loop in the subst_wildcards function. This patch fixes that,
and also adds a big documentation comment and tests.
Changes:
- Document exactly what subst_wildcards does (it's quite
complicated!)
- Spot circular recursion rather than blowing the call stack.
- Fix some odd behaviour in {eval_cmd} support.
The previous code would transform "foo {eval_cmd} echo bar" to
"bar", ignoring the "foo" prefix entirely. Similarly, it would
transform "{eval_cmd}xecho foo" to "foo" (skipping over the 'x',
which it assumed to be a space).
- Be more explicit about how values get stringified
- Make the interaction between {eval_cmd} and ignore_error=True a
more uniform.
The previous code performed partial evaluation when ignore_error
was true except when doing {eval_cmd}, when it would discard the
partially evaluated command string.
- Add some simple tests.
These correspond to documentation examples. You can run them with
pytest. Note that this doesn't add a dependency on pytest unless
you actually want to run the tests (since Python will merrily
"parse" code where the named modules aren't in scope!)
Differences in behaviour from the original code:
- Clearer eval_cmd behavior (see above)
- Circular references are now spotted. Before, a call to
subst_wildcards('{a}', {'a': '{b}', 'b': '{a}'})
would cause an infinite loop. This is the sort of thing that you
can trigger by a mistake in your config files, and what caused me
to look at the function in the first place. Now it reports an
error (regardless of the value of ignore_error).
- List items are now stringified recursively in
_stringify_wildcard_value. This makes rules like {'a': ['b', 10]}
work. The functionality isn't used in dvsim at the moment (because
all lists are lists of strings), but it's probably a bit cleaner.
- Computed wildcard names are now possible. Probably not a
particularly useful feature, but it comes for free with the
"iterate over matches from the left" implementation, so it can't
hurt to support explicitly.
Signed-off-by: Rupert Swarbrick <rswarbrick@lowrisc.org>
diff --git a/util/dvsim/utils_test.py b/util/dvsim/utils_test.py
new file mode 100644
index 0000000..d8267b7
--- /dev/null
+++ b/util/dvsim/utils_test.py
@@ -0,0 +1,77 @@
+# Copyright lowRISC contributors.
+# Licensed under the Apache License, Version 2.0, see LICENSE for details.
+# SPDX-License-Identifier: Apache-2.0
+
+'''pytest-based testing for functions in utils.py'''
+
+import os
+import pytest
+from .utils import _subst_wildcards, subst_wildcards
+
+
+def test_subst_wildcards():
+ '''Pytest-compatible test for the subst_wildcards function.'''
+ # Basic checks
+ assert subst_wildcards('foo {x} baz', {'x': 'bar'}) == 'foo bar baz'
+
+ # Stringify
+ assert (subst_wildcards('{a}, {b}, {c}, {d}',
+ {'a': 'a', 'b': True, 'c': 42, 'd': ['{b}', 10]}) ==
+ 'a, 1, 42, 1 10')
+
+ # Ignored wildcards (with or without a match in mdict)
+ assert (subst_wildcards('{a} {b}', {'a': 'aye', 'b': 'bee'},
+ ignored_wildcards=['a']) ==
+ '{a} bee')
+ assert (subst_wildcards('{a} {b}', {'b': 'bee'},
+ ignored_wildcards=['a']) ==
+ '{a} bee')
+
+ # Environment variables. We will always have PWD and can probably assume
+ # that this won't itself have any braced substrings.
+ assert (subst_wildcards('{PWD}', {}) == os.environ['PWD'])
+
+ # Missing variable with ignore_error=False, running _subst_wildcards
+ # instead so that we can catch the error. (We assume that 'biggles' isn't
+ # in the environment)
+ with pytest.raises(ValueError) as excinfo:
+ _subst_wildcards('{biggles} {b}', {'b': 'bee'}, [], False, [])
+ assert "unknown wildcard, '{biggles}'" in str(excinfo.value)
+
+ # ignore_error=True.
+ assert (subst_wildcards('{biggles} {b}', {'b': 'bee'},
+ ignore_error=True) ==
+ '{biggles} bee')
+
+ # Check we support (non-circular) recursion
+ assert (subst_wildcards('{a}', {'a': '{b}', 'b': 'c'}) == 'c')
+
+ # Check we spot circular recursion
+ with pytest.raises(ValueError) as excinfo:
+ _subst_wildcards('{a}', {'a': '{b}', 'b': '{a}'}, [], False, [])
+ assert "circular expansion of wildcard '{a}'" in str(excinfo.value)
+
+ # Check we also complain about circular recursion with ignore_error
+ with pytest.raises(ValueError) as excinfo:
+ _subst_wildcards('{a}', {'a': '{b}', 'b': '{a}'}, [], True, [])
+ assert "circular expansion of wildcard '{a}'" in str(excinfo.value)
+
+ # Computed variable names (probably not a great idea, but it's probably
+ # good to check this works the way we think)
+ assert subst_wildcards('{a}b}', {'a': 'a {', 'b': 'bee'}) == 'a bee'
+
+ # Some eval_cmd calls (using echo, which should always work)
+ assert (subst_wildcards('{eval_cmd}echo foo {b}', {'b': 'bar'}) ==
+ 'foo bar')
+
+ # Make sure that nested commands work
+ assert (subst_wildcards('{eval_cmd} {eval_cmd} echo echo a', {}) == 'a')
+
+ # Recursive expansion
+ assert (subst_wildcards('{var}',
+ {
+ 'var': '{{foo}_xyz_{bar}}',
+ 'foo': 'p',
+ 'bar': 'q',
+ 'p_xyz_q': 'baz'
+ }) == 'baz')
