API Reference¶

Base class for static groups.

A static group should be defined by subclassing StaticGroupBase. It must have type annotations to represent its child groups and rules.

class CustomStaticGroup(StaticGroupBase):
    child_rule1: Rule[str]
    child_rule2: Rule[Literal["a"]]
    child_group1: AnotherStaticGroup
    child_group2: RulesGroup

    '''
    Generic type parameters like ``[str]`` in this example is ignored
    at runtime. They are just hints for the type checker and IDE.
    '''

The child nodes are automatically instanciated when the parent is instanciated. So you can read them without assigning values:

g = CustomStaticGroup()
print(g.child_rule1)
print(g.child_group1)

Remember that the child rules are automatically instanciated but not initialized . you have to manually initialize them with Rule.init

g.child_rule1.init("child_file1.txt", copy)(souce_file, SELF)

As a design pattern, it is recommended to have an initializer method that initializes the child rules and calls the initializer of the child groups to recursively initialize all the rules in the sub-tree.

from __future__ import annotations
from pathlib import Path
from jtcmake import StaticGroupBase, Rule, SELF


class MyGroup(StaticGroupBase):
    __globals__ = globals()  # For Sphinx's doctest. Not necessary in normal situations.
    child_rule: Rule[str]
    child_group: MyChildGroup

    def init(self, text: str, repeat: int) -> MyGroup:
        # Initializer for this class. The method name "init" is not
        # reserved so you can choose your own one.

        # Initialize the direct child rules
        self.child_rule.init("<R>.txt", Path.write_text)(SELF, text)

        # Initialize the child group
        self.child_group.init(self.child_rule, repeat)

        return self


class MyChildGroup(StaticGroupBase):
    __globals__ = globals()  # For Sphinx's doctest. Not necessary in normal situations.
    foo: Rule[str]

    def init(self, src: Path, repeat: int) -> MyChildGroup:
        @self.foo.init("<R>.txt")
        def _(src=src, dst=SELF, repeat=repeat):
            text = src.read_text()
            dst.write_text(text * repeat)

        return self


g = MyGroup("out").init("abc", 2)
g.make()

assert Path("out/child_rule.txt").read_text() == "abc"
assert Path("out/child_group/foo.txt").read_text() == "abcabc"

import shutil; shutil.rmtree("out")  # Cleanup for Sphinx's doctest

Note

When you override the __init__ method, you have to call super().__init__ in it with appropriate arguments.

Parameters:

driname – directory name of this group. dirname="foo" is equivalent to prefix="foo/"
prefix – path prefix of this group. You may not specify dirname and prefix at the same time. If both dirname and prefix is none, prefix will be “”
use_default_logger – if True, logs will be printed to stderr or displayed as HTML if the code is running on Jupyter Notebook.
logfile – str, PathLike, Logger, object with a write(str) method, or list/tuple of them, indicating the target(s) to which logs should be output.

clean() → None¶: Delete all the existing files of this group.

property groups: Mapping[str, IGroup]¶

Readonly dictionary of child groups.

Key: base name of the child group
Value: child group node object

make(dry_run: bool = False, keep_going: bool = False, *, njobs: int | None = None) → MakeSummary¶

Make rules in this group and their dependencies

Parameters:

dry_run (bool) – instead of actually excuting the methods, print expected execution logs.
keep_going (bool) – If False (default), stop everything when a rule fails. If True, when a rule fails, keep executing other rules except the ones depend on the failed rule.
njobs (int) – Maximum number of rules that can be made simultaneously using multiple threads and processes. Defaults to 1 (single process, single thread).

See also

See the description of jtcmake.make for more detail of njobs

property prefix: str¶

Path prefix of this group.

Seealso:: set_prefix()

property rules: Mapping[str, Rule]¶

Readonly dictionary of child rules.

Key: base name of the child rule
Value: child rule node object

select_files(pattern: str | List[str] | Tuple[str]) → List[IFile]¶

Create list of files in this group sub-tree that match pattern.

This is the file-version of select_groups(). See its documentation for detail.

Examples

<ROOT>
|-- a(r)
|   |-- a (f:a.txt)
|   `-- b (f:b.html)
|
`-- b(g)
    `-- a(r)
        `-- a (f:a.txt)

from jtcmake import UntypedGroup, SELF

# Building the above group tree
g = UntypedGroup()  # Root group
g.add("a", { "a": "a.txt", "b": "b.html" }, lambda x,y: ())(SELF.a, SELF.b)
g.add_group("b")
g.b.add("a", { "a": "a.txt" }, lambda x: ())(SELF.a)

assert g.select_files("**/a") == [g.a.a, g.b.a.a]
assert g.select_files("a/*") == [g.a.a, g.a.b]

select_groups(pattern: str | List[str] | Tuple[str]) → List[IGroup]¶

Create list of groups in this group sub-tree that match pattern. The list may include this group itself.

Groups are gathered based on the given pattern in a manner similar to how we specify a set of files using a glob pattern on Unix.

Parameters:

pattern –

str or list/tuple of str representing a pattern of relative names of offspring nodes.

If pattern is a list/tuple of strs, it must be a sequence of base name patterns like ["a", "b", "c"] or ["a", "*b*", "c", "**"].

If pattern is a str, it will be internally translated into an equivalent list-of-str pattern by splitting it with /. So, for example, g.select_groups("a/b/c") is equivalent to ``g.select_groups(["a", "b", "c"]).

Suppose we have the following group tree (the tree may have rules as well but we omit them since this method collects groups only).

<ROOT>
|
|-- a1
|   |-- b1
|   `-- b2
|       `-- c1
|
|-- a2
|   `-- b1
|
`-- a1/b1

We use a list of strs to identify each group node. For example, the (absolute) name of the forth group from the top (the deepest one) is ["a1", "b2", "c1"]. Its relative name with respect to the group ["a1"] is ["b2", "c1"], and "c1" is its base name.

Pattern matching is basically as simple as pattern ["a1", "b1"] matches the relative name ["a1", "b1"]. Additionally, you can use wildcard * in patterns as follows.

Double stars ** can appear as a special base name indicating zero or more repetition of arbitrary base names. It may NOT appear inside a base name like a/**b/c
Single stars * can appear inside a base name indicating zero or more repetition of arbitrary character.

Examples

from jtcmake import UntypedGroup

# Building the above group tree
g = UntypedGroup()  # Root group
g.add_group("a1")
g.a1.add_group("b1")
g.a1.add_group("b2")
g.a1.b2.add_group("c1")
g.add_group("a2")
g.a2.add_group("b1")
g.add_group("a1/b1")

assert g.select_groups("a1/b1") == [g.a1.b1]
assert g.select_groups(["a1", "b1"]) == [g.a1.b1]

# In the following example,  ``/`` in the base name is treated
# as a normal character and has no effect as a base name boundary
assert g.select_groups(["a1/b1"]) == [g["a1/b1"]]

assert g.select_groups("a*") == [g.a1, g.a2, g["a1/b1"]]
assert g.select_groups("*/*1") == [g.a1.b1, g.a2.b1]
assert g.select_groups("**/*1") == [
    g.a1, g.a1.b1, g.a1.b2.c1, g.a2.b1, g["a1/b1"]
]
assert g.select_groups("**") == [
    g,  # root is included
    g.a1,
    g.a1.b1,
    g.a1.b2,
    g.a1.b2.c1,
    g.a2,
    g.a2.b1,
    g["a1/b1"],
]
assert g.a1.select_groups("*") == g.select_groups("a1/*")

Note

Current implementation collects nodes using pre-order DFS but it may be changed in the future.

select_rules(pattern: str | List[str] | Tuple[str]) → List[IRule]¶

Create list of rules in this group sub-tree that match pattern.

This is the rule-version of select_groups(). See its documentation for detail.

Examples

<ROOT>
|-- a0(r)
|-- a1
|   `-- a2(r)
`-- a3
    `-- a4(r)

from jtcmake import UntypedGroup, SELF

# Building the above group tree
g = UntypedGroup()  # Root group
g.add("a0", lambda x: ())(SELF)
g.add_group("a1")
g.a1.add("a2", lambda x: ())(SELF)
g.add_group("a3")
g.a3.add("a4", lambda x: ())(SELF)

assert g.select_rules("a*") == [g.a0]
assert g.select_rules("*/a*") == [g.a1.a2, g.a3.a4]

set_prefix(dirname: object | None = None, *, prefix: object | None = None) → T_Self¶

Set the path prefix of this group.

Parameters:

dirname – if specified, prefix will be dirname + "/"
prefix – path prefix.

You must specify either but not both of dirname or prefix.

self.set_prefix("a") is equivalent to self.set_prefix(prefix="a/").

If this group is not the root group and the given prefix is a relative path, the path prefix of the parent group will be added to its start. Absolute paths do not undergo this prefixing.

Note

This method may be called only when the prefix is not yet determined. i.e. You may NOT call this method whenever,

You have created this group as a root group
You have once called it
You have once read self.prefix: reading self.prefix internally finalizes the prefix to "{name of this group}/"
You have once read the prefix of a child group: reading a child’s prefix internally reads the parent’s prefix
You have initialized any rule in the sub-tree: initializing a rule internally reads its parent’s prefix

Example

# (For Unix only)

from jtcmake import UntypedGroup

g = UntypedGroup("root")

g.add_group("foo").set_prefix("foo-dir")  # dirname
g.add_group("bar").set_prefix(prefix="bar-")  # prefix
g.add_group("baz").set_prefix("/tmp/baz")  # dirname abspath
g.add_group("qux")  # no explicit setting

assert g.prefix == "root/"
assert g.foo.prefix == "root/foo-dir/"
assert g.bar.prefix == "root/bar-"
assert g.baz.prefix == "/tmp/baz/"
assert g.qux.prefix == "root/qux/"

touch(file: bool = True, memo: bool = True, create: bool = True, t: float | None = None) → None¶

For every rule in the group, touch (set mtime to now) the output files and force the memo to record the current input state.

Parameters:

file (bool) – if False, files won’t be touched. Defaults to True.
memo (bool) – if False, memos won’t be modified. Defaults to True.
create (bool) – if True, missing files will be created. Otherwise, only the existing files will be touched. This option has no effect with file=False.

A group that contains groups as children.

When writing type hints, children’s type can be passed as a generic type parameter like GroupsGroup[SomeGroupClass] .

from pathlib import Path
from typing import Union
from jtcmake import SELF, StaticGroupBase, GroupsGroup, Rule

class Child1(StaticGroupBase):
    __globals__ = globals()  # For Sphinx's doctest. Not necessary in normal situations.
    rule1: Rule[str]

    def init(self, text: str):
        self.rule1.init("<R>.txt", Path.write_text)(SELF, text)
        return self

class Child2(StaticGroupBase):
    __globals__ = globals()  # For Sphinx's doctest. Not necessary in normal situations.
    rule2: Rule[str]

    def init(self, text: str):
        self.rule2.init("<R>.txt", Path.write_text)(SELF, text * 2)

g: GroupsGroup[Union[Child1, Child2]] = GroupsGroup("out")

# Set the child class to use by default
g.set_default_child(Child1)

for i in range(2):
    # Child1 will be the child class
    g.add_group(f"child1-{i}").init(str(i))

for i in range(2):
    # Explicity giving the child class Child2
    g.add_group(f"child2-{i}", Child2).init(str(i))

g.make()

assert Path("out/child1-0/rule1.txt").read_text() == "0"
assert Path("out/child1-1/rule1.txt").read_text() == "1"
assert Path("out/child2-0/rule2.txt").read_text() == "00"
assert Path("out/child2-1/rule2.txt").read_text() == "11"

import shutil; shutil.rmtree("out")  # Cleanup for Sphinx's doctest

Parameters:

driname – directory name of this group. dirname="foo" is equivalent to prefix="foo/"
prefix – path prefix of this group. You may not specify dirname and prefix at the same time. If both dirname and prefix is none, prefix will be “”
use_default_logger – if True, logs will be printed to stderr or displayed as HTML if the code is running on Jupyter Notebook.
logfile – str, PathLike, Logger, object with a write(str) method, or list/tuple of them, indicating the target(s) to which logs should be output.

add_group(name: str, child_group_type: Type[T_Child] | None = None) → T_Child¶

Append a child group to this group.

Parameters:

name (str) – name of the new child group.
child_group_type – class of the new child group. If not specified, and if the default child group class is available (set by self.set_default_child), it will be used. Otherwise, an exception will be raised.

clean() → None¶: Delete all the existing files of this group.

property groups: Mapping[str, T_Child]¶

Readonly dictionary of child groups.

Key: base name of the child group
Value: child group node object

make(dry_run: bool = False, keep_going: bool = False, *, njobs: int | None = None) → MakeSummary¶

Make rules in this group and their dependencies

Parameters:

dry_run (bool) – instead of actually excuting the methods, print expected execution logs.
keep_going (bool) – If False (default), stop everything when a rule fails. If True, when a rule fails, keep executing other rules except the ones depend on the failed rule.
njobs (int) – Maximum number of rules that can be made simultaneously using multiple threads and processes. Defaults to 1 (single process, single thread).

See also

See the description of jtcmake.make for more detail of njobs

property prefix: str¶

Path prefix of this group.

Seealso:: set_prefix()

property rules: Mapping[str, Rule]¶

Readonly dictionary of child rules.

Key: base name of the child rule
Value: child rule node object

select_files(pattern: str | List[str] | Tuple[str]) → List[IFile]¶

Create list of files in this group sub-tree that match pattern.

This is the file-version of select_groups(). See its documentation for detail.

Examples

<ROOT>
|-- a(r)
|   |-- a (f:a.txt)
|   `-- b (f:b.html)
|
`-- b(g)
    `-- a(r)
        `-- a (f:a.txt)

from jtcmake import UntypedGroup, SELF

# Building the above group tree
g = UntypedGroup()  # Root group
g.add("a", { "a": "a.txt", "b": "b.html" }, lambda x,y: ())(SELF.a, SELF.b)
g.add_group("b")
g.b.add("a", { "a": "a.txt" }, lambda x: ())(SELF.a)

assert g.select_files("**/a") == [g.a.a, g.b.a.a]
assert g.select_files("a/*") == [g.a.a, g.a.b]

select_groups(pattern: str | List[str] | Tuple[str]) → List[IGroup]¶

Create list of groups in this group sub-tree that match pattern. The list may include this group itself.

Groups are gathered based on the given pattern in a manner similar to how we specify a set of files using a glob pattern on Unix.

Parameters:

pattern –

str or list/tuple of str representing a pattern of relative names of offspring nodes.

If pattern is a list/tuple of strs, it must be a sequence of base name patterns like ["a", "b", "c"] or ["a", "*b*", "c", "**"].

If pattern is a str, it will be internally translated into an equivalent list-of-str pattern by splitting it with /. So, for example, g.select_groups("a/b/c") is equivalent to ``g.select_groups(["a", "b", "c"]).

Suppose we have the following group tree (the tree may have rules as well but we omit them since this method collects groups only).

<ROOT>
|
|-- a1
|   |-- b1
|   `-- b2
|       `-- c1
|
|-- a2
|   `-- b1
|
`-- a1/b1

We use a list of strs to identify each group node. For example, the (absolute) name of the forth group from the top (the deepest one) is ["a1", "b2", "c1"]. Its relative name with respect to the group ["a1"] is ["b2", "c1"], and "c1" is its base name.

Pattern matching is basically as simple as pattern ["a1", "b1"] matches the relative name ["a1", "b1"]. Additionally, you can use wildcard * in patterns as follows.

Double stars ** can appear as a special base name indicating zero or more repetition of arbitrary base names. It may NOT appear inside a base name like a/**b/c
Single stars * can appear inside a base name indicating zero or more repetition of arbitrary character.

Examples

from jtcmake import UntypedGroup

# Building the above group tree
g = UntypedGroup()  # Root group
g.add_group("a1")
g.a1.add_group("b1")
g.a1.add_group("b2")
g.a1.b2.add_group("c1")
g.add_group("a2")
g.a2.add_group("b1")
g.add_group("a1/b1")

assert g.select_groups("a1/b1") == [g.a1.b1]
assert g.select_groups(["a1", "b1"]) == [g.a1.b1]

# In the following example,  ``/`` in the base name is treated
# as a normal character and has no effect as a base name boundary
assert g.select_groups(["a1/b1"]) == [g["a1/b1"]]

assert g.select_groups("a*") == [g.a1, g.a2, g["a1/b1"]]
assert g.select_groups("*/*1") == [g.a1.b1, g.a2.b1]
assert g.select_groups("**/*1") == [
    g.a1, g.a1.b1, g.a1.b2.c1, g.a2.b1, g["a1/b1"]
]
assert g.select_groups("**") == [
    g,  # root is included
    g.a1,
    g.a1.b1,
    g.a1.b2,
    g.a1.b2.c1,
    g.a2,
    g.a2.b1,
    g["a1/b1"],
]
assert g.a1.select_groups("*") == g.select_groups("a1/*")

Note

Current implementation collects nodes using pre-order DFS but it may be changed in the future.

select_rules(pattern: str | List[str] | Tuple[str]) → List[IRule]¶

Create list of rules in this group sub-tree that match pattern.

This is the rule-version of select_groups(). See its documentation for detail.

Examples

<ROOT>
|-- a0(r)
|-- a1
|   `-- a2(r)
`-- a3
    `-- a4(r)

from jtcmake import UntypedGroup, SELF

# Building the above group tree
g = UntypedGroup()  # Root group
g.add("a0", lambda x: ())(SELF)
g.add_group("a1")
g.a1.add("a2", lambda x: ())(SELF)
g.add_group("a3")
g.a3.add("a4", lambda x: ())(SELF)

assert g.select_rules("a*") == [g.a0]
assert g.select_rules("*/a*") == [g.a1.a2, g.a3.a4]

set_default_child(default_child_group_type: Type[T_Child]) → GroupsGroup[T_Child]¶: Sets the default child class, which will be used when GroupsGroup.add_group() is called with child_group_type unspecified.

set_prefix(dirname: object | None = None, *, prefix: object | None = None) → T_Self¶

Set the path prefix of this group.

Parameters:

dirname – if specified, prefix will be dirname + "/"
prefix – path prefix.

You must specify either but not both of dirname or prefix.

self.set_prefix("a") is equivalent to self.set_prefix(prefix="a/").

If this group is not the root group and the given prefix is a relative path, the path prefix of the parent group will be added to its start. Absolute paths do not undergo this prefixing.

Note

This method may be called only when the prefix is not yet determined. i.e. You may NOT call this method whenever,

You have created this group as a root group
You have once called it
You have once read self.prefix: reading self.prefix internally finalizes the prefix to "{name of this group}/"
You have once read the prefix of a child group: reading a child’s prefix internally reads the parent’s prefix
You have initialized any rule in the sub-tree: initializing a rule internally reads its parent’s prefix

Example

# (For Unix only)

from jtcmake import UntypedGroup

g = UntypedGroup("root")

g.add_group("foo").set_prefix("foo-dir")  # dirname
g.add_group("bar").set_prefix(prefix="bar-")  # prefix
g.add_group("baz").set_prefix("/tmp/baz")  # dirname abspath
g.add_group("qux")  # no explicit setting

assert g.prefix == "root/"
assert g.foo.prefix == "root/foo-dir/"
assert g.bar.prefix == "root/bar-"
assert g.baz.prefix == "/tmp/baz/"
assert g.qux.prefix == "root/qux/"

set_props(default_child_group_type: Type[T_Child] | None = None, dirname: StrOrPath | None = None, prefix: StrOrPath | None = None) → GroupsGroup[T_Child]¶: Convenient method that works as set_default_child() and set_prefix combined.

touch(file: bool = True, memo: bool = True, create: bool = True, t: float | None = None) → None¶

For every rule in the group, touch (set mtime to now) the output files and force the memo to record the current input state.

Parameters:

file (bool) – if False, files won’t be touched. Defaults to True.
memo (bool) – if False, memos won’t be modified. Defaults to True.
create (bool) – if True, missing files will be created. Otherwise, only the existing files will be touched. This option has no effect with file=False.

A group that contains rules as children.

from pathlib import Path
from jtcmake import RulesGroup, SELF

g = RulesGroup("out")

for i in range(3):
    g.add(f"child{i}.txt", Path.write_text)(SELF, str(i))

g.make()

assert Path("out/child0.txt").read_text() == "0"
assert Path("out/child1.txt").read_text() == "1"
assert Path("out/child2.txt").read_text() == "2"

import shutil; shutil.rmtree("out")  # Cleanup for Sphinx's doctest

Parameters:

driname – directory name of this group. dirname="foo" is equivalent to prefix="foo/"
prefix – path prefix of this group. You may not specify dirname and prefix at the same time. If both dirname and prefix is none, prefix will be “”
use_default_logger – if True, logs will be printed to stderr or displayed as HTML if the code is running on Jupyter Notebook.
logfile – str, PathLike, Logger, object with a write(str) method, or list/tuple of them, indicating the target(s) to which logs should be output.

add(name: object, outs: object | None = None, method: object | None = None, /, *, noskip: bool = False) → Callable[[...], object]¶

Create a temporary function to add a rule to this group.

This method works similarly to Rule.init(). See its documentation for details.

Parameters:

name – name of the rule.
output_files – if not specified, name will be used.
method – function to create the output files

Returns:

If method is provided, it returns a function rule_adder, whose signature is the same as the given method. Calling it as rule_adder(*args, **kwargs) appends a new rule to the group.

If method is not provided, it returns a decorator function method_decorator, which consumes a function and appends a new rule whose method is the given function.

While executing this rule, method is called as method(*args, **kwargs).

Example

With method provided:

from __future__ import annotations
from pathlib import Path
from jtcmake import RulesGroup, SELF, VFile, File

g = RulesGroup("out")

def split_write(text: str, file1: Path, file2: Path):
    # Write first half of ``text`` to file1 and the rest to file2
    n = len(text)
    file1.write_text(text[: n // 2])
    file2.write_text(text[n // 2: n])

def cat(srcs: list[Path], dst: Path):
    with dst.open("w") as f:
        f.writelines(src.read_text() for src in srcs)

# File path may be str or PathLike
g.add("foo", {"a": "a.txt", "b": Path("b.txt")}, split_write)("abcd", SELF[0], SELF[1])

g.add("bar", ["x.txt", VFile("y.txt")], split_write)("efgh", SELF[0], SELF[1])

g.add("baz", "baz.txt", cat)([g.foo[0], g.foo[1], g.bar[0], g.bar[1]], SELF)

# file paths of str or PathLike (excluding File/VFile) are
# internally converted to File
assert isinstance(g.bar["x.txt"], File)

# file paths of VFile remains VFile
assert isinstance(g.bar["y.txt"], VFile)

g.make()

assert Path("out/a.txt").read_text() == "ab"
assert Path("out/b.txt").read_text() == "cd"
assert Path("out/x.txt").read_text() == "ef"
assert Path("out/y.txt").read_text() == "gh"
assert Path("out/baz.txt").read_text() == "abcdefgh"

Without method:

from __future__ import annotations
from pathlib import Path
from jtcmake import RulesGroup, SELF, VFile, File

g = RulesGroup("out")

@g.add("foo")
def foo(dst: Path = SELF):
    dst.write_text("abc")

@g.add("bar")
def bar(dst: Path = SELF):
    dst.write_text("xyz")

@g.add("baz")
def baz(dst: Path = SELF, srcs: list[Path] = [g.foo, g.bar]):
    with dst.open("w") as f:
        f.writelines(src.read_text() for src in srcs)

g.make()

assert Path("out/foo").read_text() == "abc"
assert Path("out/bar").read_text() == "xyz"
assert Path("out/baz").read_text() == "abcxyz"

addvf(name: object, outs: object | None = None, method: object | None = None, /, *, noskip: bool = False) → Callable[[...], object]¶

Create a temporary function to add a rule to this group.

This method is equal to self.add except the default file class is VFile instead of File.

See the documentation of self.add for more information.

clean() → None¶: Delete all the existing files of this group.

property groups: Mapping[str, IGroup]¶

Readonly dictionary of child groups.

Key: base name of the child group
Value: child group node object

make(dry_run: bool = False, keep_going: bool = False, *, njobs: int | None = None) → MakeSummary¶

Make rules in this group and their dependencies

Parameters:

dry_run (bool) – instead of actually excuting the methods, print expected execution logs.
keep_going (bool) – If False (default), stop everything when a rule fails. If True, when a rule fails, keep executing other rules except the ones depend on the failed rule.
njobs (int) – Maximum number of rules that can be made simultaneously using multiple threads and processes. Defaults to 1 (single process, single thread).

See also

See the description of jtcmake.make for more detail of njobs

property prefix: str¶

Path prefix of this group.

Seealso:: set_prefix()

property rules: Mapping[str, Rule]¶

Readonly dictionary of child rules.

Key: base name of the child rule
Value: child rule node object

select_files(pattern: str | List[str] | Tuple[str]) → List[IFile]¶

Create list of files in this group sub-tree that match pattern.

This is the file-version of select_groups(). See its documentation for detail.

Examples

<ROOT>
|-- a(r)
|   |-- a (f:a.txt)
|   `-- b (f:b.html)
|
`-- b(g)
    `-- a(r)
        `-- a (f:a.txt)

from jtcmake import UntypedGroup, SELF

# Building the above group tree
g = UntypedGroup()  # Root group
g.add("a", { "a": "a.txt", "b": "b.html" }, lambda x,y: ())(SELF.a, SELF.b)
g.add_group("b")
g.b.add("a", { "a": "a.txt" }, lambda x: ())(SELF.a)

assert g.select_files("**/a") == [g.a.a, g.b.a.a]
assert g.select_files("a/*") == [g.a.a, g.a.b]

select_groups(pattern: str | List[str] | Tuple[str]) → List[IGroup]¶

Create list of groups in this group sub-tree that match pattern. The list may include this group itself.

Groups are gathered based on the given pattern in a manner similar to how we specify a set of files using a glob pattern on Unix.

Parameters:

pattern –

str or list/tuple of str representing a pattern of relative names of offspring nodes.

If pattern is a list/tuple of strs, it must be a sequence of base name patterns like ["a", "b", "c"] or ["a", "*b*", "c", "**"].

If pattern is a str, it will be internally translated into an equivalent list-of-str pattern by splitting it with /. So, for example, g.select_groups("a/b/c") is equivalent to ``g.select_groups(["a", "b", "c"]).

Suppose we have the following group tree (the tree may have rules as well but we omit them since this method collects groups only).

<ROOT>
|
|-- a1
|   |-- b1
|   `-- b2
|       `-- c1
|
|-- a2
|   `-- b1
|
`-- a1/b1

We use a list of strs to identify each group node. For example, the (absolute) name of the forth group from the top (the deepest one) is ["a1", "b2", "c1"]. Its relative name with respect to the group ["a1"] is ["b2", "c1"], and "c1" is its base name.

Pattern matching is basically as simple as pattern ["a1", "b1"] matches the relative name ["a1", "b1"]. Additionally, you can use wildcard * in patterns as follows.

Double stars ** can appear as a special base name indicating zero or more repetition of arbitrary base names. It may NOT appear inside a base name like a/**b/c
Single stars * can appear inside a base name indicating zero or more repetition of arbitrary character.

Examples

from jtcmake import UntypedGroup

# Building the above group tree
g = UntypedGroup()  # Root group
g.add_group("a1")
g.a1.add_group("b1")
g.a1.add_group("b2")
g.a1.b2.add_group("c1")
g.add_group("a2")
g.a2.add_group("b1")
g.add_group("a1/b1")

assert g.select_groups("a1/b1") == [g.a1.b1]
assert g.select_groups(["a1", "b1"]) == [g.a1.b1]

# In the following example,  ``/`` in the base name is treated
# as a normal character and has no effect as a base name boundary
assert g.select_groups(["a1/b1"]) == [g["a1/b1"]]

assert g.select_groups("a*") == [g.a1, g.a2, g["a1/b1"]]
assert g.select_groups("*/*1") == [g.a1.b1, g.a2.b1]
assert g.select_groups("**/*1") == [
    g.a1, g.a1.b1, g.a1.b2.c1, g.a2.b1, g["a1/b1"]
]
assert g.select_groups("**") == [
    g,  # root is included
    g.a1,
    g.a1.b1,
    g.a1.b2,
    g.a1.b2.c1,
    g.a2,
    g.a2.b1,
    g["a1/b1"],
]
assert g.a1.select_groups("*") == g.select_groups("a1/*")

Note

Current implementation collects nodes using pre-order DFS but it may be changed in the future.

select_rules(pattern: str | List[str] | Tuple[str]) → List[IRule]¶

Create list of rules in this group sub-tree that match pattern.

This is the rule-version of select_groups(). See its documentation for detail.

Examples

<ROOT>
|-- a0(r)
|-- a1
|   `-- a2(r)
`-- a3
    `-- a4(r)

from jtcmake import UntypedGroup, SELF

# Building the above group tree
g = UntypedGroup()  # Root group
g.add("a0", lambda x: ())(SELF)
g.add_group("a1")
g.a1.add("a2", lambda x: ())(SELF)
g.add_group("a3")
g.a3.add("a4", lambda x: ())(SELF)

assert g.select_rules("a*") == [g.a0]
assert g.select_rules("*/a*") == [g.a1.a2, g.a3.a4]

set_prefix(dirname: object | None = None, *, prefix: object | None = None) → T_Self¶

Set the path prefix of this group.

Parameters:

dirname – if specified, prefix will be dirname + "/"
prefix – path prefix.

You must specify either but not both of dirname or prefix.

self.set_prefix("a") is equivalent to self.set_prefix(prefix="a/").

If this group is not the root group and the given prefix is a relative path, the path prefix of the parent group will be added to its start. Absolute paths do not undergo this prefixing.

Note

This method may be called only when the prefix is not yet determined. i.e. You may NOT call this method whenever,

You have created this group as a root group
You have once called it
You have once read self.prefix: reading self.prefix internally finalizes the prefix to "{name of this group}/"
You have once read the prefix of a child group: reading a child’s prefix internally reads the parent’s prefix
You have initialized any rule in the sub-tree: initializing a rule internally reads its parent’s prefix

Example

# (For Unix only)

from jtcmake import UntypedGroup

g = UntypedGroup("root")

g.add_group("foo").set_prefix("foo-dir")  # dirname
g.add_group("bar").set_prefix(prefix="bar-")  # prefix
g.add_group("baz").set_prefix("/tmp/baz")  # dirname abspath
g.add_group("qux")  # no explicit setting

assert g.prefix == "root/"
assert g.foo.prefix == "root/foo-dir/"
assert g.bar.prefix == "root/bar-"
assert g.baz.prefix == "/tmp/baz/"
assert g.qux.prefix == "root/qux/"

touch(file: bool = True, memo: bool = True, create: bool = True, t: float | None = None) → None¶

For every rule in the group, touch (set mtime to now) the output files and force the memo to record the current input state.

Parameters:

file (bool) – if False, files won’t be touched. Defaults to True.
memo (bool) – if False, memos won’t be modified. Defaults to True.
create (bool) – if True, missing files will be created. Otherwise, only the existing files will be touched. This option has no effect with file=False.

A group that have groups and rules as children.

Note

Type annotation for this class is weak and you won’t get much support from static type checkers and IDEs. It is recommended to use StaticGroupBase, GroupsGroup, and RulesGroup when writing a long code.

from pathlib import Path
from jtcmake import UntypedGroup, SELF, Rule, StaticGroupBase

def add1(src: Path, dst: Path):
    dst.write_text(str(int(src.read_text()) + 1))

g = UntypedGroup("out")

@g.add("rule0")
def _write_0(p: Path = SELF):
    p.write_text("0")

g.add("rule1", add1)(g.rule0, SELF)

# ``add_group`` with ``child_group_type=None`` adds an UntypedGroup
g.add_group("group1")

g.group1.add("rule2", add1)(g.rule1, SELF)

class Child(StaticGroupBase):
    __globals__ = globals()  # For Sphinx's doctest. Not necessary in normal situations.
    rule: Rule

g.add_group("group2", Child)

g.group2.rule.init("rule3", add1)(g.group1.rule2, SELF)

g.make()

assert Path("out/rule0").read_text() == "0"
assert Path("out/rule1").read_text() == "1"
assert Path("out/group1/rule2").read_text() == "2"
assert Path("out/group2/rule3").read_text() == "3"

import shutil; shutil.rmtree("out")  # Cleanup for Sphinx's doctest

Parameters:

driname – directory name of this group. dirname="foo" is equivalent to prefix="foo/"
prefix – path prefix of this group. You may not specify dirname and prefix at the same time. If both dirname and prefix is none, prefix will be “”
use_default_logger – if True, logs will be printed to stderr or displayed as HTML if the code is running on Jupyter Notebook.
logfile – str, PathLike, Logger, object with a write(str) method, or list/tuple of them, indicating the target(s) to which logs should be output.

add(name: object, outs: object | None = None, method: object | None = None, /, *, noskip: bool = False) → Callable[[...], object]¶

Create a temporary function to add a rule to this group.

This method works similarly to Rule.init(). See its documentation for details.

Parameters:

name – name of the rule.
output_files – if not specified, name will be used.
method – function to create the output files

Returns:

If method is provided, it returns a function rule_adder, whose signature is the same as the given method. Calling it as rule_adder(*args, **kwargs) appends a new rule to the group.

If method is not provided, it returns a decorator function method_decorator, which consumes a function and appends a new rule whose method is the given function.

While executing this rule, method is called as method(*args, **kwargs).

Example

With method provided:

from __future__ import annotations
from pathlib import Path
from jtcmake import RulesGroup, SELF, VFile, File

g = RulesGroup("out")

def split_write(text: str, file1: Path, file2: Path):
    # Write first half of ``text`` to file1 and the rest to file2
    n = len(text)
    file1.write_text(text[: n // 2])
    file2.write_text(text[n // 2: n])

def cat(srcs: list[Path], dst: Path):
    with dst.open("w") as f:
        f.writelines(src.read_text() for src in srcs)

# File path may be str or PathLike
g.add("foo", {"a": "a.txt", "b": Path("b.txt")}, split_write)("abcd", SELF[0], SELF[1])

g.add("bar", ["x.txt", VFile("y.txt")], split_write)("efgh", SELF[0], SELF[1])

g.add("baz", "baz.txt", cat)([g.foo[0], g.foo[1], g.bar[0], g.bar[1]], SELF)

# file paths of str or PathLike (excluding File/VFile) are
# internally converted to File
assert isinstance(g.bar["x.txt"], File)

# file paths of VFile remains VFile
assert isinstance(g.bar["y.txt"], VFile)

g.make()

assert Path("out/a.txt").read_text() == "ab"
assert Path("out/b.txt").read_text() == "cd"
assert Path("out/x.txt").read_text() == "ef"
assert Path("out/y.txt").read_text() == "gh"
assert Path("out/baz.txt").read_text() == "abcdefgh"

Without method:

from __future__ import annotations
from pathlib import Path
from jtcmake import RulesGroup, SELF, VFile, File

g = RulesGroup("out")

@g.add("foo")
def foo(dst: Path = SELF):
    dst.write_text("abc")

@g.add("bar")
def bar(dst: Path = SELF):
    dst.write_text("xyz")

@g.add("baz")
def baz(dst: Path = SELF, srcs: list[Path] = [g.foo, g.bar]):
    with dst.open("w") as f:
        f.writelines(src.read_text() for src in srcs)

g.make()

assert Path("out/foo").read_text() == "abc"
assert Path("out/bar").read_text() == "xyz"
assert Path("out/baz").read_text() == "abcxyz"

add_group(name: str, child_group_type: Type[T_Child]) → T_Child¶

add_group(name: str) → UntypedGroup

Append a child group to this group.

Parameters:

name (str) – name of the new child group.
child_group_type – class of the new child group. If not specified, UntypedGroup will be used.

addvf(name: object, outs: object | None = None, method: object | None = None, /, *, noskip: bool = False) → Callable[[...], object]¶

Create a temporary function to add a rule to this group.

This method is equal to self.add except the default file class is VFile instead of File.

See the documentation of self.add for more information.

clean() → None¶: Delete all the existing files of this group.

property groups: Mapping[str, IGroup]¶

Readonly dictionary of child groups.

Key: base name of the child group
Value: child group node object

make(dry_run: bool = False, keep_going: bool = False, *, njobs: int | None = None) → MakeSummary¶

Make rules in this group and their dependencies

Parameters:

dry_run (bool) – instead of actually excuting the methods, print expected execution logs.
keep_going (bool) – If False (default), stop everything when a rule fails. If True, when a rule fails, keep executing other rules except the ones depend on the failed rule.
njobs (int) – Maximum number of rules that can be made simultaneously using multiple threads and processes. Defaults to 1 (single process, single thread).

See also

See the description of jtcmake.make for more detail of njobs

property prefix: str¶

Path prefix of this group.

Seealso:: set_prefix()

property rules: Mapping[str, Rule]¶

Readonly dictionary of child rules.

Key: base name of the child rule
Value: child rule node object

select_files(pattern: str | List[str] | Tuple[str]) → List[IFile]¶

Create list of files in this group sub-tree that match pattern.

This is the file-version of select_groups(). See its documentation for detail.

Examples

<ROOT>
|-- a(r)
|   |-- a (f:a.txt)
|   `-- b (f:b.html)
|
`-- b(g)
    `-- a(r)
        `-- a (f:a.txt)

from jtcmake import UntypedGroup, SELF

# Building the above group tree
g = UntypedGroup()  # Root group
g.add("a", { "a": "a.txt", "b": "b.html" }, lambda x,y: ())(SELF.a, SELF.b)
g.add_group("b")
g.b.add("a", { "a": "a.txt" }, lambda x: ())(SELF.a)

assert g.select_files("**/a") == [g.a.a, g.b.a.a]
assert g.select_files("a/*") == [g.a.a, g.a.b]

select_groups(pattern: str | List[str] | Tuple[str]) → List[IGroup]¶

Create list of groups in this group sub-tree that match pattern. The list may include this group itself.

Groups are gathered based on the given pattern in a manner similar to how we specify a set of files using a glob pattern on Unix.

Parameters:

pattern –

str or list/tuple of str representing a pattern of relative names of offspring nodes.

If pattern is a list/tuple of strs, it must be a sequence of base name patterns like ["a", "b", "c"] or ["a", "*b*", "c", "**"].

If pattern is a str, it will be internally translated into an equivalent list-of-str pattern by splitting it with /. So, for example, g.select_groups("a/b/c") is equivalent to ``g.select_groups(["a", "b", "c"]).

Suppose we have the following group tree (the tree may have rules as well but we omit them since this method collects groups only).

<ROOT>
|
|-- a1
|   |-- b1
|   `-- b2
|       `-- c1
|
|-- a2
|   `-- b1
|
`-- a1/b1

We use a list of strs to identify each group node. For example, the (absolute) name of the forth group from the top (the deepest one) is ["a1", "b2", "c1"]. Its relative name with respect to the group ["a1"] is ["b2", "c1"], and "c1" is its base name.

Pattern matching is basically as simple as pattern ["a1", "b1"] matches the relative name ["a1", "b1"]. Additionally, you can use wildcard * in patterns as follows.

Double stars ** can appear as a special base name indicating zero or more repetition of arbitrary base names. It may NOT appear inside a base name like a/**b/c
Single stars * can appear inside a base name indicating zero or more repetition of arbitrary character.

Examples

from jtcmake import UntypedGroup

# Building the above group tree
g = UntypedGroup()  # Root group
g.add_group("a1")
g.a1.add_group("b1")
g.a1.add_group("b2")
g.a1.b2.add_group("c1")
g.add_group("a2")
g.a2.add_group("b1")
g.add_group("a1/b1")

assert g.select_groups("a1/b1") == [g.a1.b1]
assert g.select_groups(["a1", "b1"]) == [g.a1.b1]

# In the following example,  ``/`` in the base name is treated
# as a normal character and has no effect as a base name boundary
assert g.select_groups(["a1/b1"]) == [g["a1/b1"]]

assert g.select_groups("a*") == [g.a1, g.a2, g["a1/b1"]]
assert g.select_groups("*/*1") == [g.a1.b1, g.a2.b1]
assert g.select_groups("**/*1") == [
    g.a1, g.a1.b1, g.a1.b2.c1, g.a2.b1, g["a1/b1"]
]
assert g.select_groups("**") == [
    g,  # root is included
    g.a1,
    g.a1.b1,
    g.a1.b2,
    g.a1.b2.c1,
    g.a2,
    g.a2.b1,
    g["a1/b1"],
]
assert g.a1.select_groups("*") == g.select_groups("a1/*")

Note

Current implementation collects nodes using pre-order DFS but it may be changed in the future.

select_rules(pattern: str | List[str] | Tuple[str]) → List[IRule]¶

Create list of rules in this group sub-tree that match pattern.

This is the rule-version of select_groups(). See its documentation for detail.

Examples

<ROOT>
|-- a0(r)
|-- a1
|   `-- a2(r)
`-- a3
    `-- a4(r)

from jtcmake import UntypedGroup, SELF

# Building the above group tree
g = UntypedGroup()  # Root group
g.add("a0", lambda x: ())(SELF)
g.add_group("a1")
g.a1.add("a2", lambda x: ())(SELF)
g.add_group("a3")
g.a3.add("a4", lambda x: ())(SELF)

assert g.select_rules("a*") == [g.a0]
assert g.select_rules("*/a*") == [g.a1.a2, g.a3.a4]

set_prefix(dirname: object | None = None, *, prefix: object | None = None) → T_Self¶

Set the path prefix of this group.

Parameters:

dirname – if specified, prefix will be dirname + "/"
prefix – path prefix.

You must specify either but not both of dirname or prefix.

self.set_prefix("a") is equivalent to self.set_prefix(prefix="a/").

If this group is not the root group and the given prefix is a relative path, the path prefix of the parent group will be added to its start. Absolute paths do not undergo this prefixing.

Note

This method may be called only when the prefix is not yet determined. i.e. You may NOT call this method whenever,

You have created this group as a root group
You have once called it
You have once read self.prefix: reading self.prefix internally finalizes the prefix to "{name of this group}/"
You have once read the prefix of a child group: reading a child’s prefix internally reads the parent’s prefix
You have initialized any rule in the sub-tree: initializing a rule internally reads its parent’s prefix

Example

# (For Unix only)

from jtcmake import UntypedGroup

g = UntypedGroup("root")

g.add_group("foo").set_prefix("foo-dir")  # dirname
g.add_group("bar").set_prefix(prefix="bar-")  # prefix
g.add_group("baz").set_prefix("/tmp/baz")  # dirname abspath
g.add_group("qux")  # no explicit setting

assert g.prefix == "root/"
assert g.foo.prefix == "root/foo-dir/"
assert g.bar.prefix == "root/bar-"
assert g.baz.prefix == "/tmp/baz/"
assert g.qux.prefix == "root/qux/"

touch(file: bool = True, memo: bool = True, create: bool = True, t: float | None = None) → None¶

For every rule in the group, touch (set mtime to now) the output files and force the memo to record the current input state.

Parameters:

file (bool) – if False, files won’t be touched. Defaults to True.
memo (bool) – if False, memos won’t be modified. Defaults to True.
create (bool) – if True, missing files will be created. Otherwise, only the existing files will be touched. This option has no effect with file=False.

class jtcmake.Rule(*args: object, **kwargs: object)¶

clean() → None¶: Delete all the existing files of this rule.

property files: T¶

Readonly dictionary of output files.

Key: file key of the file
Value: jtcmake.IFile object

init(method: Callable[[P], None], /, *, noskip: bool = False) → Callable[[P], Rule]¶

Create a temporary function to complete initialization of this rule.

Note

This method must be called only for uninitialized rules which do not have the output files, method, and method’s arguments assigned to themselvs yet.

For example, you have to call this method for rules of StaticGroupBase-like groups while you must not for rules owned by RulesGroup groups.

Parameters:

output_files –
if not specified, this rule’s name will be used. The following three forms are accepted.
- Dict ({"key1": file1, "key2": file2, ...}): key1, key2, … are the file keys and file1, file2, … are the file paths.
- List ([file1, file2, ...]): equivalent to a dict-form of {str(file1): file1, str(file2): file2}
- Atom (file): equivalent to a dict-form of {str(file): file}
File keys must be str. File paths may be either str or PathLike including File and VFile. If a given file path is neither File or VFile, it will be converted to File by File(file_path).
method – function to create the output files

Returns:

rule_initializer, a temporary function whose signature is the same as the given method. Calling it as rule_adder(*args, **kwargs) completes initialization of this rule.

While executing this rule, method is called as method(*args, **kwargs).

Hint

Name Reference in File Paths

A file path in a dict-form output_files may contain text symbols "<R>" and "<F>", which will be replaced with the rule’s name and the corresponding file key, respectively.

For example, output_files={ "foo": Path("<R>-<F>.txt") } for a rule named “myrule” is equivalent to output_files={ "foo": Path("myrule-foo.txt") }.

In list/atom form of output_files, you may use <R> too but <F> is not allowed because the file keys are derived from the file paths.

Path Prefixing

If given as a relative path, file paths get transformed by adding the parent group’s path prefix to the head.

For example, if the parent group’s path prefix is "output_dir/", output_files of { "a": File("a.txt") } will be transformed into { "a": File("out/a.txt") }.

You can suppress this conversion by passing the path as an absolute path.

Rule-initializer and Argument Substitution

Rule.init(output_files, method) returns a temporary function, rule_initializer and you must further call it with the arguments to be eventually passed to method like:

g.rule.init(output_files, method)(SELF, foo="bar")

Examples

Basic usage:

from __future__ import annotations
from pathlib import Path
from typing import Literal
from jtcmake import StaticGroupBase, Rule, SELF

def split_write(text: str, file1: Path, file2: Path):
    # Write first half of ``text`` to file1 and the rest to file2
    n = len(text)
    file1.write_text(text[: n // 2])
    file2.write_text(text[n // 2: n])


def cat(dst: Path, *srcs: Path):
    with dst.open("w") as f:
        f.writelines(src.read_text() for src in srcs)


class MyGroup(StaticGroupBase):
    __globals__ = globals()  # Only for Sphinx's doctest. Not necessary in normal situations.
    foo: Rule[Literal["a", "b"]]
    bar: Rule[str]
    buz: Rule[str]

    def init(self) -> MyGroup:
        '''
        Supplying keys other than "a" and "b" would be marked as
        a type error by type checkers such as Mypy and Pyright.
        '''
        self.foo.init({"a": "a.txt", "b": "b.txt"}, split_write)(
            "abcd", SELF[0], SELF[1]
        )

        '''
        The list below will be translated into `{"x.txt": "x.txt", "y.txt": "y.txt"}`
        (and then `{"x.txt": File("out/x.txt"), "y.txt": File("out/y.txt"}`)
        '''
        self.bar.init(["x.txt", "y.txt"], split_write)(
            "efgh", file1=SELF[0], file2=SELF[1]  # you can use keyword args
        )

        self.buz.init("w.txt", cat)(
            SELF, self.foo[0], self.foo[1], self.bar[0], self.bar[1]
        )

        return self

g = MyGroup("out").init()

g.make()

assert Path("out/a.txt").read_text() == "ab"
assert Path("out/b.txt").read_text() == "cd"
assert Path("out/x.txt").read_text() == "ef"
assert Path("out/y.txt").read_text() == "gh"
assert Path("out/w.txt").read_text() == "abcdefgh"

import shutil; shutil.rmtree("out")  # Cleanup for Sphinx's doctest

initvf(method: Callable[[P], None], /, *, noskip: bool = False) → Callable[[P], Rule]¶

Create a temporary function to initialize this rule.

This method is equal to init() except the default class constructor is VFile instead of File.

Seealso:: init()

make(*args, **kwargs) → MakeSummary¶

Make this rule and its dependencies

Parameters:

dry_run – instead of actually excuting the methods, print expected execution logs.
keep_going – If False (default), stop everything when a rule fails. If True, when a rule fails, keep executing other rules except the ones depend on the failed rule.
njobs – Maximum number of rules that can be made concurrently. Defaults to 1 (single process, single thread).

Seealso:: jtcmake.make()

property memo_value: object¶: Returns: object to be memoized

property parent: IGroup¶: Parent group node of this rule.

touch(file: bool = True, memo: bool = True, create: bool = True, t: float | None = None) → None¶

Touch (set mtime to now) the output files and force the memo to record the current input state.

Parameters:

file (bool) – if False, the output files won’t be touched. Defaults to True.
memo (bool) – if False, the memo won’t be modified. Defaults to True.
create (bool) – if True, missing files will be created. Otherwise, only the existing files will be touched. This option has no effect with file=False.

property xfiles: T¶: List of path of the input files.

class jtcmake.File(*args, **kwargs)¶

Bases: PosixPath, IFile

An instance of this class represents a file, which can be an input or output of rules.

When used as an input, on judging whether the rule must be updated, its modification time is compared to the modification time of the output files. If any of the output files is older than the input, the rule must be updated.

class jtcmake.VFile(*args, **kwargs)¶

Bases: PosixPath, IFile, IMemoAtom

An instance of this class represents a value file, which can be an input or output of rules.

When used as an input, on judging whether the rule must be updated, its modification time is compared to the modification time of the output files. If any of the output files is older than the input, the rule must be updated.

class jtcmake.IFile(*args, **kwargs)¶

Bases: Path, IAtom

Abstract base class to represent a file object.

jtcmake.make(*rule_or_groups: IGroup | IRule, dry_run: bool = False, keep_going: bool = False, njobs: int | None = None) → MakeSummary¶

make rules

Parameters:

rules_or_groups (Sequence[RuleNodeBase|Group]) – Rules and Groups containing target Rules
dry_run – instead of actually excuting methods, print expected execution logs.
keep_going – If False (default), stop everything when a rule fails. If True, when a rule fails, keep executing other rules except the ones depend on the failed rule.
njobs – Maximum number of rules that can be made concurrently. Defaults to 1 (single process, single thread).

Warning

Safely and effectively using njobs >= 2 require a certain level of understanding of Python’s threading and multiprocessing and their complications.

Only inter-process transferable rules are executed on child processes. Other rules are executed on threads of the main process, thus subject to the constraints of global interpreter lock (GIL).

inter-process transferable means being able to be sent to a child process without errors.

Child processes are started by the ‘spawn’ method, not ‘fork’, even on Linux systems.

njobs >= 2 may not work on interactive interpreters. It should work on Jupyter Notebook/Lab but any function or class that are defined on the notebook is not inter-process transferable.

jtcmake.print_graphviz(target_nodes: IGroup | IRule | list[Union[jtcmake.group_tree.core.IGroup, jtcmake.group_tree.core.IRule]], output_file: str | PathLike[str] | None = None, max_dependency_depth: int = 1000000, *, rankdir: Literal['TD', 'LR'] = 'LR')¶

Visualize the dependency graph using Graphviz. Graphviz binaries are required to be available in PATH.

Parameters:

group – Group node whose Rules will be visualized
output_file –
If specified, graph will be written into the file. Otherwise, graph will be printed to the terminal (available on Jupyter only). Graph format depends on the file extension:
- .svg: SVG
- .htm or .html: HTML (SVG image embedded)
- .dot: Graphviz’s DOT code (text)

jtcmake.print_mermaid(target_nodes: GroupTreeNode | list[GroupTreeNode], output_file: StrOrPath | None = None, max_dependency_depth: int = 1000000, direction: Direction = 'LR', mermaidjs: str | None = 'https://unpkg.com/mermaid@9.2.2/dist/mermaid.js')¶

Visualizes the structure of a group tree and dependency of its rules using mermaid.js.

Example

import shutil
import jtcmake as jtc

g = jtc.UntypedGroup("root")
g.add("a", shutil.copy)(jtc.File("src1.txt"), SELF)
g.add("b", shutil.copy)(jtc.File("src2.txt"), SELF)
g.add("c", shutil.copy)(jtc.File("src3.txt"), SELF)

print_method(g, "graph.html")  # visualize the whole tree
print_method([g.a, g.c], "graph.html")  # visualize specific nodes

jtcmake.print_method(rule: IRule)¶: Show how the method of the given rule will be called.