# The debputy manifest format

_This is [reference guide] and is primarily useful if you have an idea of what you are looking for._
_If you are new to `debputy`, maybe you want to read [GETTING-STARTED-WITH-dh-debputy.md](GETTING-STARTED-WITH-dh-debputy.md) first._

<!-- To writers and reviewers: Check the documentation against https://documentation.divio.com/ -->


## Prerequisites

This guide assumes familiarity with Debian packaging in general.  Notably, you should understand
the different between a (Debian) source package and a (Debian) binary package (e.g., `.deb`) plus
how these concepts relates to `debian/control` (the source control file).

Additionally, the reader is expected to have an understanding of globs and substitution variables.

It is probably very useful to have an understanding on how a binary package is
assembled.  While `debputy` handles the assembly for you, this document will not go in details
with this. Prior experience with `debhelper` (notably the `dh`-style `debian/rules`) may be
useful but should not be a strict requirement.


# The basic manifest

The manifest is a YAML document with a dictionary at the top level layer.  You can choose to leave
for YAML version be implicit.  All manifests must include a manifest version, which will enable
the format to change over time.  For now, there is only one version (`0.1`) and you have to include
the line:

    manifest-version: "0.1"


On its own, the manifest containing only `manifest-version: "..."` will not do anything.  So if you
end up only having the `manifest-version` key in the manifest, you can just remove the manifest and
rely entirely on the built-in rules.


# Path matching rules

Most of the manifest is about declaring rules for a given path such as "foo must be a symlink"
or "bar must be owned by root:tty and have mode 02755".  

The manifest contains the following types of matches:

 1) Exact path matches.  These specify the path inside the debian package exactly without any
    form of wildcards (e.g., `*` or `?`).  However, they can use substitution variables.
    Examples include:
    * `usr/bin/sudo`
    * `usr/lib/{{DEB_HOST_MULTIARCH}}/libfoo.so`

    Having a leading `/` is optional.  I.e. `/usr/bin/sudo` and `usr/bin/sudo` are considered the
    same path.

 2) Glob based path matches.  These specify a rule that match any path that matches a given
    glob.  These rules must contain a glob character (e.g., `*`) _and_ a `/`. Examples include:

    * `foo/*`
    * `foo/*.txt`
    * `usr/lib/{{DEB_HOST_MULTIARCH}}/lib*.so*`

    Note that if the glob does not contain a `/`, then it falls into the Basename glob rule
    below.

 3) Basename glob matches.  These specify a rule that match any path where the basename matches
    a given glob.  They must contain a glob character (e.g., `*`) but either must not have a
    `/` in them at all or start with `**/` and not have any `/` from there.
    Examples include:

    * `"*.la"`
    * `"**/*.md"`

    The examples use explicit quoting because YAML often interprets `*` as an anchor rule in the
    places where you are likely to use this kind of glob.  The use of the `**/`-prefix is
    optional. 

However, there are also cases where these matching rules can cause conflicts.  This is covered
in the [Conflict resolution](#Conflict resolution) section below.


## Limitations on debputy path rules

Path rules:

 1) Must match relatively to the package root.
 2) Never resolves a symlink.
    * If `bin/ls` is a symlink, then `bin/ls` matches the symlink (and never the target)
    * If `bin` is a symlink, then `bin/ls` (or `bin/*`) would fail to match, because
      matching would require resolving the symlink.

These limitations are in place because of implementation details in `debputy`.


## Conflict resolution

The manifest can contain seemly mutually exclude rules.  As an example, if you ask for
`foo/symlink` to be a symlink but also state that you want to exclude `foo` entirely
from the debian package  then the manifest now has two mutually exclusive requests.

To resolve these problems, `debputy` relies on the following rules for conflict resolutions:

  1. If two requests are mutually exclusive (and not resolvable by ordering), the manifest is invalid
     and `debputy` rejects it with an error. This is the case if you define `foo` to be a directory
     *AND* a symlink at the same time. Here `debputy` will force you to change at least one
     of the definitions.

  2. Requests are loosely-speaking ranked and applied in order:
     1. `installations` (ordered)
     2. `transformations` (ordered)
     3. `path-metadata` (unordered), `create-symlinks` (unordered), `directories` (unordered)
     
     This has two important effects:
      * If the manifest that has a transformation rule and a `create-symlinks` rule applying to the
        same path, the package will end up with a symlink as the `create-symlinks` is applied
        after the `transformations`.
      * On the flip side, `transformations` will apply *before* globs are resolved in later rules.
        If you have an `exclude` transformation and a glob match for a `path-metadata`, then the
        `path-metadata` will not match any path excluded by the `exclude` transformation.

  3. The transformation rules inside `transformations` are ordered and are applied in the listed
     order. Due to the explicit ordering, transformation rules inside `transformations` generally
     do not trigger other conflict resolution rules.

     Note keep in mind that all transformations must apply to at least one path. So even with
     the manual ordering, it is still possible to cause conflicts (albeit a different kind).

  4. When two path matches (that are not otherwise ordered) match the same path, `debputy` weights
     the path matches by the number of non-glob characters. If two globs still tie after this,
     the longest prefix of non-glob tie breaks.  Examples of this:

     * `foo/*.txt` has a weight of (8, 4).  I.e., 8 non-glob characters, but only 4 of them
       is in the prefix.
     * `foo/bar*` has a weight of (7, 7).  I.e., 7 non-glob characters and they are all used
       in the prefix.
     * `foo/b*xt` has weight (7, 5). I.e., 7 non-glob characters, but only 5 of them
       is in the prefix.

     These 3 patterns are listed in the order they are tried.  The `foo/*.txt` pattern is 
     strongest because it has 8 non-glob in total characters and the others only have 7.
     The `foo/bar*` pattern is stronger than `foo/b*xt` because longest prefix is used as
     tiebreaker and `foo/bar*` has prefix length 7 vs. the 5 from `foo/b*xt`.

     Note: One caveat with this rule is that `debputy` normalizes patterns to an internal
     format with a fixed prefix and the weights are done on that. The normalization means
     that `foo/*` vs. `/foo/*` vs. `./foo/*` are all read the same and therefore has the
     same weight.

     Some notably (but hopefully unsurprising) special cases:
     * If one path match is an exact path and the other one is a glob match, then the exact
       match is stronger even in the extreme case where they tie on all the match length 
       numbers.  I.e., `foo` is stronger than `foo*` because the former is not a glob but
       the latter is. 

     * If both path matches are exact matches, then this generally triggers the 
       `mutual exclusive request` case and triggers a hard error (see above).


## All definitions must be used

Whenever you define a request or rule in the manifest, `debputy` will insist on it being used
at least once.  The exception to this rule being conditional rules where the condition
evaluates to `false` (in which case the rule never used and does not trigger an error).

This is useful for several reasons:

 1. The definition may have been shadowed by another rule and would have been ignored otherwise
 2. The definition may no longer be useful, but its present might confuse a future reader of
    the manifest.

In all cases, `debputy` will tell you if a definition was unused and where you can find that
definition.


## debputy globs

In general, the following rules applies to globs in `debputy`.

 * The `*` match 0 or more times any characters except `/`.
 * The `?` match exactly one character except `/`.
 * The glob `foo/*` matches _everything_ inside `foo/` including hidden files (i.e., paths starting 
   with `.`) unlike `bash`/`sh` globs. However, `foo/*` does _not_ match `foo/` itself (this latter
   part matches `bash`/`sh` behaviour and should be unsurprising).
 * For the special-cases where `**` is supported, then `**` matches zero or more levels of directories.
   This means that `foo/**/*` match any path beneath `foo` (but still not `foo`).  This is mostly relevant
   for built-in path matches as it is currently not possible to define `foo/**/...` patterns in the manifest.


# Substitution rules

The `debputy` tool supports substitution in various places (usually paths) via the following
rules.  That means:

 1) All substitutions must start with `{{` and end with `}}`.  The part inbetween is
    the `VARIABLE` and must match the regular expression `[A-Za-z0-9][-_:0-9A-Za-z]*`.
    Note that you can use space around the variable name if you feel that increases readability.
    (That is, `{{ FOO }}` can be used as an alternative to `{{FOO}}`).
 2) The `VARIABLE` will be result from a set of builtin variables and the variables from
    `dpkg-architecture`.
 3) You can use `{{DOUBLE_OPEN_CURLY_BRACE}}` and `{{DOUBLE_CLOSE_CURLY_BRACE}}` (built-in variables)
    if you want a literal `{{` or `}}`  would otherwise have triggered an undesired expansion.
 4) All `{{VARIABLE}}` must refer to a defined variable.
 5) There are no expression syntax inside the `{{ ... }}` (unlike jinja2 and other template languages).
    This rule may be changed in the future (with a new manifest version).

Keep in mind that substitution _cannot_ be used everywhere.  There are specific places where
it can be used.  Also, substitution _cannot be used_ to introduce globs into paths.  When a
substitution occurs inside a path all characters inserted are treated as literal characters.


## Builtin or common substitution variables

 * `{{NEWLINE}}` or `{{NL}}` expands to a literal newline (LF) `\n`.
 * `{{TAB}}` expands to a literal tab `\t`.
 * `{{OPEN_CURLY_BRACE}}`  / `{{CLOSE_CURLY_BRACE}}` expand to `{` / `}`
 * `{{DOUBLE_OPEN_CURLY_BRACE}}` / `{{DOUBLE_CLOSE_CURLY_BRACE}}` expands to `{{` / `}}`.
 * `{{PACKAGE}}` expands to the binary package name of the current binary package. This substitution
   only works/applies when the substitution occurs in the context of a concrete binary package.
 * Plus any of the variables from produced by `dpkg-architecture`, such as `{{DEB_HOST_MULTIARCH}}`.

The following variables from `/usr/share/dpkg/pkg-info.mk` (`dpkg`) are also available:

 * DEB_SOURCE (as `{{DEB_SOURCE}}`)
 * DEB_VERSION (as `{{DEB_VERSION}}`)
 * DEB_VERSION_EPOCH_UPSTREAM (as `{{DEB_VERSION_EPOCH_UPSTREAM}}`)
 * DEB_VERSION_UPSTREAM_REVISION (as `{{DEB_VERSION_UPSTREAM_REVISION}}`)
 * DEB_VERSION_UPSTREAM (as `{{DEB_VERSION_UPSTREAM}}`)
 * SOURCE_DATE_EPOCH (as `{{SOURCE_DATE_EPOCH}}`)

These have the same definition as those from the `dpkg` provided makefile.


# Restrictions on defining ownership of paths

In some parts of the manifest, you can specify which user or group should have ownership of
a given path.  As an example, you can define a directory to be owned by `root` with group `tty`
(think `chmod root:tty <some/path>`).

Ownership is generally defined via the keys `owner` and `group`.  For each of them, you can use
one of the following formats:

 1) A name (e.g., `owner: root`).
 2) An id (e.g., `owner: 0`). Please avoid using quotes around the ID in YAML.
 3) A name and an id with a colon inbetween (e.g., `owner: "root:0"`).  The name must always
    come first here.  You may have to quote the value to prevent the YAML parser from being
    confused.

All three forms are valid and provide the same result.  Unless you have a compelling reason to
pick a particular form, the name-only is recommended for simplicity.  Notably, it does not
require your co-maintainer or future you to remember what the IDs mean.

Regardless of which form you pick:

 1) The provided owner must be defined by Debian `base-passwd` file, which are the only users guaranteed
    to be present on every Debian system.
    * Concretely, `debputy` verifies the input against `/usr/share/base-passwd/passwd.master` and
      `/usr/share/base-passwd/group.master` (except for `root` / `0` as an optimization).

 2) If the `name:id` form is used, then the name and the id values must match.  I.e., `root:2` would
    be invalid as the id for `root` is defined to be `0` in the `base-passwd` data files.

 3) The `debputy` tool maintains a `deny`-list of owners that it refuses even though `base-passwd`
    defines them. As a notable non-exhaustive example, `debputy` considers `nobody` or id `65534` 
    (the ID of `nobody` / `nogroup`) to be invalid owners.


# Conditional rules

There are cases, where a given rule should only apply in certain cases - such as only when a given
build profile is active (`DEB_BUILD_PROFILES` / `dpkg-buildpackage -P`).  For rules that
*support being conditional*, the condition is generally defined via the `when:` key and the condition
is then described beneath the `when:`.

As an example:

    packages:
        util-linux:
            create-symlinks:
            - path: sbin/agetty
              target: /sbin/getty
              when:
                  # On Hurd, the package "hurd" ships "sbin/getty".
                  arch-matches: '!hurd-any'


When the condition under `when:` resolves to `true`, the rule will and must be used.  When the
condition resolves to `false`, the rule will not be applied even if it could have been.  Note that
conditions are *not* used as a conflict resolution and as such two conditional rules can still
cause conflicts even though their conditions are mutually exclusive.  This may be changed in a
later version of `debputy` provided `debputy` can assert the two conditions ensures the rules are
mutually exclusive.

The `when:` key has either a mapping, a list or a string as value depending on the condition.
Each supported condition is described in the following subsections.

## Package architecture match condition `arch-matches` (mapping)

The `arch-matches` condition is used to assert whether the content of the package is for a given
architecture.  The `arch-matches` condition cannot be applied to architecture independent packages
(`Architecture: all`) as the answer would not change between build and is therefore confusing and
redundant at best.

The `arch-matches` condition accounts for cross-compiling and will therefore match against
`DEB_HOST_ARCH` by default.

As an example:

    packages:
        util-linux:
            create-symlinks:
            - path: sbin/agetty
              target: /sbin/getty
              when:
                  # On Hurd, the package "hurd" ships "sbin/getty".
                  arch-matches: '!hurd-any'

The `arch-matches` must be defined as a part of a mapping, where `arch-matches` is the key.  The
value is a string and will be the architecture restriction as a space separated list of architecture
names or architecture wildcards (same syntax as the architecture restriction in `Build-Depends`
in `debian/control` except there is no enclosing `[]` brackets). The names/wildcards can optionally
be prefixed by `!` to negate them.  However, either *all* names / wildcards must have negation or
*none* of them may have it.

## Active build profile match condition `build-profiles-matches` (mapping)

The `build-profiles-matches` condition is used to assert whether the active build profiles
(`DEB_BUILD_PROFILES` / `dpkg-buildpackage -P`) matches a given build profile restriction.

As an example:

    # TODO: Not the best example (`create-symlinks` is an unlikely use-case for this condition)
    packages:
        foo:
            create-symlinks:
            - path: sbin/agetty
              target: /sbin/getty
              when:
                  build-profiles-matches: '<!pkg.foo.mycustomprofile>'

The `build-profiles-matches` must be defined as a part of a mapping, where `build-profiles-matches`
is the key.  The value is a string using the same syntax as the `Build-Profiles` field from `debian/control`
(i.e., a space separated list of `<[!]profile ...>` groups).

## Can run produced binaries `can-execute-compiled-binaries` (string)

The `can-execute-compiled-binaries` condition is used to assert the build can assume
that all compiled binaries can be run as-if they were native binaries. For native
builds, this condition always evaluates to `true`.  For cross builds, the condition
is generally evaluates to `false`.  However, there are special-cases where binaries
can be run during cross-building. Accordingly, this condition is subtly different
from the `cross-compiling` condition.

Note this condition should *not* be used when you know the binary has been built
for the build architecture (`DEB_BUILD_ARCH`) or for determining whether build-time tests
should be run (for build-time tests, please use the `run-build-time-tests` condition instead).
Some upstream build systems are advanced enough to distinguish building a final product vs.
building a helper tool that needs to run during build.  The latter will often be compiled by
a separate compiler (often using `$(CC_FOR_BUILD)`, `cc_for_build` or similar variable names
in upstream build systems for that compiler).

As an example:

    # TODO: Not the best example (`create-symlinks` is an unlikely use-case for this condition)
    packages:
        foo:
            create-symlinks:
            - path: sbin/agetty
              target: /sbin/getty
              # Only for native builds or when we can transparently run a compiled
              when: can-execute-compiled-binaries

The `can-execute-compiled-binaries` condition is specified as a string.

## Cross-Compiling condition `cross-compiling` (string)

The `cross-compiling` condition is used to determine if the current build is performing a cross
build (i.e., `DEB_BUILD_GNU_TYPE` != `DEB_HOST_GNU_TYPE`). Often this has consequences for what
is possible to do.

Note if you specifically want to know:

 * whether build-time tests should be run, then please use the `run-build-time-tests` condition.
 * whether compiled binaries can be run as if it was a native binary, please use the
   `can-execute-compiled-binaries` condition instead.  That condition accounts for cross-building
   in its evaluation.

As an example:

    # TODO: Not the best example (`create-symlinks` is an unlikely use-case for this condition)
    packages:
        foo:
            create-symlinks:
            - path: sbin/agetty
              target: /sbin/getty
              when: cross-compiling

The `cross-compiling` condition is specified as a string.

## Whether build time tests should be run `run-build-time-tests` (string)

The `run-build-time-tests` condition is used to determine whether (build time) tests should
be run for this build.  This condition roughly translates into whether `nocheck` is present
in `DEB_BUILD_OPTIONS`.

In general, the manifest *should not* prevent build time tests from being run during cross-builds.

As an example:

    # TODO: Not the best example (`create-symlinks` is an unlikely use-case for this condition)
    packages:
        foo:
            create-symlinks:
            - path: sbin/agetty
              target: /sbin/getty
              when: run-build-time-tests

The `run-build-time-tests` condition is specified as a string.

## Negated condition `not` (mapping)

It is possible to negate a condition via the `not` condition.

As an example:

    packages:
        util-linux:
            create-symlinks:
            - path: sbin/getty
              target: /sbin/agetty
              when:
                  # On Hurd, the package "hurd" ships "sbin/getty".
                  # This example is an alternative to `arch-marches: '!hurd-any`
                  not:
                      arch-matches: 'hurd-any'

The `not` condition is specified as a mapping, where the key is `not` and the
value is a nested condition.

## All or any of a list of conditions `allOf`/`anyOf` (list)

It is possible to aggregate conditions using the `allOf` or `anyOf` condition. This provide
`X and Y` and `X or Y` semantics (respectively).

As an example:

    packages:
        util-linux:
            create-symlinks:
            - path: sbin/getty
              target: /sbin/agetty
              when:
                  # Only ship getty on linux except for s390(x)
                  allOf:
                      - arch-matches: 'linux-any'
                      - arch-matches: '!s390 !s390x'

The `allOf` and `anyOf` conditions are specified as lists, where each entry is a nested condition.
The lists need at least 2 entries as with fewer entries the `allOf` and `anyOf` conditions are
redundant.

# Binary package rules

Inside the manifest, the `packages` mapping can be used to define requests for the binary packages
you want `debputy` to produce.  Each key inside `packages` must be the name of a binary package
defined in `debian/control`.  The value is a dictionary defining which features that `debputy`
should apply to that binary package.  An example could be:


    packages:
        foo:
            create-symlinks:
            - path: usr/share/foo/my-first-symlink
              target: /usr/share/bar/symlink-target
            - path: usr/lib/${DEB_HOST_MULTIARCH}/my-second-symlink
              target: /usr/lib/${DEB_HOST_MULTIARCH}/baz/symlink-target
       bar:
            directories:
            - some/empty/directory
            - another/empty/directory
            - path: a/third-empty/directory
              owner: www-data
              group: www-data

In this case, `debputy` will create some symlinks inside the `foo` package and some directories for
the `bar` package.  The following subsections define the keys you can use under each binary package.

## Installations

For source packages building a single binary, the `dh_auto_install` from debhelper will default to
providing everything from upstream's install in the binary package.  The `debputy` tool matches this
behaviour and accordingly, the `installations` feature is only relevant in this case when you need to
manually specify something upstream's install did not cover.

For sources, that build multiple binaries, where `dh_auto_install` does not detect anything to install,
or when `dh_auto_install --destdir debian/tmp` is used, the `installations` section of the manifest is
used to declare what goes into which binary package. An example:

    installations:
      - install:
            sources: "usr/bin/foo"
            into: foo
      - install:
            sources: "usr/*"
            into: foo-extra


All installation rules are processed in order (top to bottom).  Once a path has been matched, it can
no longer be matched by future rules.  In the above example, then `usr/bin/foo` would be in the `foo`
package while everything in `usr` *except* `usr/bin/foo` would be in `foo-extra`.  If these had been
ordered in reverse, the `usr/bin/foo` rule would not have matched anything and caused `debputy`
to reject the input as an error on that basis.  This behaviour is similar to "DEP-5" copyright files,
except the order is reversed ("DEP-5" uses "last match wins", where here we are doing "first match wins")

In the rare case that some path need to be installed into two packages at the same time, then this is
generally done by changing `into` into a list of packages.

All installations are currently run in *source* package context.  This implies that:

  1) No package specific substitutions are available. Notably `{{PACKAGE}}` cannot be resolved.
  2) All conditions are evaluated in source context.  For 99.9% of users, this makes no difference,
     but there is a cross-build feature that changes the "per package" architecture which is affected.

This is a limitation that should be fixed in `debputy`.

**Attention debhelper users**: Note the difference between `dh_install` (etc.) vs. `debputy` on
overlapping matches for installation.

**Limitations**: Note due to an implementation detail, the patterns in the `installation` section is
currently a bit more limited than elsewhere.

## Install rule search directories

Most install rules apply their patterns against search directories such as `debian/tmp` by default.

The default search directory order (highest priority first) is:

 1) The upstream install directory (usually, `debian/tmp`)
 2) The source package root directory (`.`)

Each search directory is tried in order.  When a pattern matches an entry in a search directory (even
if that entry is reserved by another package), further search directories will *not* be tried. As an example,
consider the pattern `usr/bin/foo*` and the files:

  `SOURCE_ROOT/debian/tmp/usr/bin/foo.sh`
  `SOURCE_ROOT/usr/bin/foo.pl`

Here the pattern will only match `SOURCE_ROOT/debian/tmp/usr/bin/foo.sh` and not `SOURCE_ROOT/usr/bin/foo.pl`.

## Generic install (`install`)

The generic `install` rule can be used to install arbitrary paths into packages and is *similar* to how
`dh_install` from debhelper works.  It is a two "primary" uses.

  1) The classic "install into directory" similar to the standard `dh_install`
  2) The "install as" similar to `dh-exec`'s `=>` feature.

Examples:

    installations:
      - install:
            sources: "usr/bin/tool-a"
            dest-dir: "usr/bin"
            into: foo
      - install:
            sources: "usr/bin/tool-b"
            # Implicit dest-dir: "usr/bin
            into: foo-extra
      - install:
            sources: "usr/bin/tool-c.sh"
            # Rename as a part of installing.
            as: "usr/bin/tool-c"
            into: foo-util


The value for `install` is a mapping with the following key/value pairs:

 * `sources` (required): A path match or a list of path matches defining the source path(s) to be installed.
   The path match(es) can use globs and substitutions.  Each match is tried against default search directories.
    - When a symlink is matched, then the symlink (not its target) is installed as-is.  When a directory is
     matched, then the directory is installed along with all the contents that have not already been installed
     somewhere.

 * `into` (required): Either a package name or a list of package names.

 * `dest-dir` (optional): A path defining the destination *directory*.  The value *cannot* use globs, but can
   use substitution.  If neither `as` nor `dest-dir` is given, then `dest-dir` defaults to the directory name
   of the `source`.

 * `as` (optional): A path defining the path to install the source as. This is a full path.  This option is
   mutually exclusive with `dest-dir`.  When `as` is given, then `source` must match exactly one path.  For
   this reason, `source` must contain exactly one path match when `as` is used (and said match must expand
   to exactly one "not yet installed" path).

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules). 


## Install documentation and examples (`install-docs` and `install-examples`)

These install rules are related to documentation and resemble that of `dh_installdoc` / `dh_installexamples`
respectively.  They are shorthands over the generic `install` rule with the following key features:

 1) They pre-defined the `dest-dir`.
 2) They come with pre-defined conditional logic for skipping these rules under `DEB_BUILD_OPTIONS=nodoc`,
    so you do not have to write that conditional yourself.

With these two things in mind, they behave just like the `install` rule.

Examples:

    installations:
      - install-docs:
            sources:
              - "docs/README.some-topic.md"
              - "docs/README.another-topic.md"
            into: foo
      - install-examples:
            sources: "examples/*"
            into: foo


The value for `install-docs` / `install-examples` is a mapping with the following key/value pairs:

 * `sources` (required): A path match or a list of path matches defining the source path(s) to be installed.
   The path match(es) can use globs and substitutions.  Each match is tried against default search directories.
    - When a symlink is matched, then the symlink (not its target) is installed as-is.  When a directory is
     matched, then the directory is installed along with all the contents that have not already been installed
     somewhere.

   * **CAVEAT**:  Specifying `source: examples` where `examples` resolves to a directory for `install-examples`
     will give you an `examples/examples` directory in the package, which is rarely what you want. Often, you
     can solve this by using `examples/*` instead. Similar for `install-docs` and a `doc` or `docs` directory.

 * `into` (required): Either a package name or a list of package names.

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules).  This condition will
   be combined with the built-in condition provided by these rules (rather than replacing it).

Note: While the canonical name for these rules use plural, the `install-doc` and `install-example` variants
are accepted as alternative names.

## Transformations

You can define a `transformations` under the package definition, which is a list a transformation
rules.  An example:

    packages:
        foo:
            transformations:
              - exclude: '*.la'
              - move:
                    source: bar/*
                    target: foo/


### Exclude transformation rule

In some cases, the upstream build system installs files or directories that is _not_ relevant for the
Debian package (but would be relevant for other distros or for non-distro local builds).  Common examples
include `INSTALL` files or `LICENSE` files (when they are just a subset of `debian/copyright`).

In the manifest, you can ask `debputy` to exclude paths from the debian package by using the `exclude`
transformation rule.  An example being:

    packages:
        foo:
            transformations:
              - exclude: '*.la'

The value for the `exclude:` is a string.  The string is interpreted as a path or glob to be
excluded.  You can use substitutions inside the path.  If a directory is matched by an exclusion rule,
then everything beneath it is also excluded.

Note that exclusion removes paths from future glob matches and transformation rules.

### Move transformation rule

In some cases, it is necessary to rename (or relocate) paths inside the debian package.  This can be done
with the `move` transformation rule, which is a rough emulation of the `mv` command line tool.  An example being:

    packages:
        foo:
            transformations:
              - move:
                    source: bar/*
                    target: foo/
              # Note this example leaves bar/ as an empty directory. Real world usage would probably
              # follow this with an exclude rule for bar/ to avoid an empty directory

The value for `move` is a mapping with the following key/value pairs:

 * `source` (required): A path match defining the source path(s) to be renamed.  The value can use globs and
   substitutions.

 * `target` (required): A path defining the target path.  The value *cannot* use globs, but can use substitution.
   If the target ends with a literal `/` (prior to substitution), the target will *always* be a directory.

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules)

There are two basic cases:
 1. If `source` match exactly one path and `target` is not a directory (including it does not exist),
    then `target` is excluded (if it existed) and `source` is renamed to `target`.
    - If the `target` is a directory, and you intentionally want to replace it with a non-directory, please
      add an explicit `exclude` transformation for the directory prior to this transformation.

 2. If `source` match more than one path *or* `target` is a directory (or is specified with a trailing slash),
    then all matching sources a moved into `target` retaining their basename.  If `target` already contains
    overlapping basenames with `source`, then the transformation rule with abort with an error if the overlap
    contains a directory. Otherwise, any overlapping paths in `target` will be implicitly excluded first.

    - If the replacement of a directory is intentional, please add an explicit `exclude` rule for it first.
    - In the case that the `source` glob is exotic enough to match two distinct paths with the same basename,
      then `debputy` will reject the transformation on account of it being ambiguous.

In either case, parent directories of `target` is created as necessary as long as it does not trigger a conflict
(or require traversing symlinks or non-directories).  Additionally, the paths matched by `source` will no longer
match anything (since they are now renamed/relocated), which may affect what path future path matches will apply
to.

Note that like `mv`, while the `source` paths are no longer present, their parent directories will remain.

## Symlinks

Often, the upstream build system will provide the symlinks for you.  However, in some cases, it is useful for
the packager to define distribution specific symlinks. This can be done via the `create-symlinks` key.  An
example of how to do this is:

    packages:
        foo:
            create-symlinks:
            - path: usr/share/foo/my-first-symlink
              target: /usr/share/bar/symlink-target
            - path: usr/lib/${DEB_HOST_MULTIARCH}/my-second-symlink
              target: /usr/lib/${DEB_HOST_MULTIARCH}/baz/symlink-target

The value for the `create-symlinks` key is a list of dictionaries and each dictionary entry contains
the following keys:

 * `path` (required): The path that should be a symlink.  The path may contain substitution variables
   such as `${DEB_HOST_MULTIARCH}` but _cannot_ use globs.  Parent directories are implicitly created
   as necessary.
   * Note that if that path has a different definition in the file system, the file system definition is ignored.
     This means you can use this to overwrite a file or a directory with a symlink. 

 * `target` (required): Where the symlink should point to. The target may contain substitution variables
   such as `${DEB_HOST_MULTIARCH}` but _cannot_ use globs.  The link target is _not_ required to exist inside
   the package.
   * The `debputy` tool will normalize the target according to the rules of the Debian Policy.  Use absolute
     or relative target at your own preference.

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules)


## Directories

In most cases, upstream build systems and `debputy` will create all the relevant directories.  However, in some
rare cases you may want to explicitly define a path to be a directory.  Maybe to silence a linter that is
warning you about a directory being empty, to help resolve a conflict in `debputy`, or maybe you need an
empty directory that nothing else is creating for you.  In this case, you can use the `directories` key
in the manifest.  An example being:

    packages:
        bar:
            directories:
            - some/empty/directory
            - another/empty/directory
            - path: a/third-empty/directory
              owner: www-data
              group: www-data

The value for the `directories` key is a list of either directory path names or dictionaries describing
the paths.  If a list entry is a string, it is a read as a shorthand for `path: <value>`.  Each directory
entry can use the following keys:

 * `path` (required): The path that should be a directory.  The path may contain substitution variables
   such as `${DEB_HOST_MULTIARCH}` but _cannot_ use globs.  Parent directories are implicitly created
   as necessary if not explicitly listed in the `directories`.
 * `owner` (optional, default `root`): Denotes the owner of the directory (but _not_ what is inside the directory).
 * `group` (optional, default `root`): Denotes the group of the directory (but _not_ what is inside the directory).
 * `mode` (optional, default `"0755"`): Denotes the mode of the directory (but _not_ what is inside the directory).
   Note that numeric mode must always be given as a string (i.e., with quotes).
 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules)

Unless you have a specific need for the directory form, you are recommended to use the shorthand form.

Note that implicitly created directories always have owner `root:root` and mode `"0755"`.  If you need a directory
tree with special ownership/mode, you will have to list all the directories in that tree explicitly with the
relevant attributes.

## Ensure path metadata

The `debputy` normalizes the path metadata (such as ownership and mode) similar to `dh_fixperms`.
For most packages, the default is what you want.  However, in some cases, the package has a special
case or two that `debputy` does not cover.  In that case, you can tell `debputy` to use the metadata
you want by using the `path-metadata` key.  An example being:

    packages:
        foo:
            path-metadata:
              usr/bin/sudo:
                  mode: "0755"
              usr/bin/write:
                  group: tty


The value for the `path-metadata` key is a dictionary.  Each key defines a path or a glob that may
contain substitution variables.  The value for each key is a dictionary that defines the metadata
with the following keys:

 * `owner` (optional, default `root`): Denotes the owner of the paths matched by the glob/path defined
   in the key. 
 * `group` (optional, default `root`): Denotes the group of the paths matched by the glob/path defined
   in the key. 
 * `mode` (optional, default is to preserve file system mode): Denotes the mode of the paths matched
   by the glob/path defined in the key.  Note that numeric mode must always be given as a string (i.e., 
   with quotes).
 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules)

These definitions do _not_ apply recursively to directories.  If a directory is matched by a glob but
its contents are not, then the metadata will only affect the directory.

Note that `path-metadata` only affects paths in the package root directory and not virtual paths defined
by other rules in the manifest.  As an example, if you want to alter the owner or mode of a directory
created by `debputy` (per the [Directories](#Directories) section), then you have to set the modes directly
in the `directories` section rather than using a `path-metadata` rule.

## Custom binary version

In the *rare* case that you need a binary package to have a custom version, you can use the `binary-version:`
key to describe the desired package version.  An example being:

    packages:
        foo:
            # The foo package needs a different epoch because we took it over from a different
            # source package with higher epoch version
            binary-version: '1:${DEB_VERSION_UPSTREAM_REVISION}'

Use this feature sparingly as it is generally not possible to undo as each version must be monotonously
higher than the previous one.  This feature translates into `-v` option for `dpkg-gencontrol`.

The value for the `binary-version` key is a string that defines the binary version.  Generally, you will
want it to contain one of the versioned related substitution variables such as 
`${DEB_VERSION_UPSTREAM_REVISION}`.  Otherwise, you will have to remember to bump the version manually
with each upload as versions cannot be reused.


[reference documentation]: https://documentation.divio.com/reference/
