Currently, dotbot uses os.path.expandvars. However, in some cases, it may be desireable to follow proper shell parameter substitution. (see: the related manual) Here is an example usage: I want my zshrc to work. Some systems may define $ZDOTDIR to be $XDG_DATA_HOME/zsh, some may leave it undefined. The typical solution would be to use ${ZDOTDIR:-$HOME}/.zshrc as the install location. However, without parameter substitution, there is no way to achieve this.

Would it make sense to add a flag to only run the clean, link, or shell sections of the config? Sometimes I dont want the shell commands to run so I just comment them out. This could be implemented using flags (dotbot -c/--clean -l/--link -s/--shell), or possibly as "sub-commands" (dotbot {clean,link,shell} or dotbot run {clean,link,shell}) and the default would still be to perform all three tasks.

I also thought it might be useful to add another optional argument to shell configurations to prompt the user if they want the shell command to run or not, something like "confirm: true". Not really sure that would be useful if I could just tell dotbot to only clean and link.

Here is my current install.conf.yaml file:

#!/usr/bin/env bash

- clean: ['~']

set -e

- link:
  ~/.tmux.conf: tmux.conf
  ~/.vim: vim/
  ~/.vimrc: vimrc
  ~/.zsh: zsh/
  ~/.zshrc: zshrc

- shell:
  - [git submodule update --init --recursive, Installing/updating submodules]

Here is my install script:

#!/usr/bin/env bash

CONFIG="install.conf.yaml"
DOTBOT_DIR="dotbot"

DOTBOT_BIN="bin/dotbot"
BASEDIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

cd "${BASEDIR}"
git submodule update --init --recursive "${DOTBOT_DIR}"

"${BASEDIR}/${DOTBOT_DIR}/${DOTBOT_BIN}" -d "${BASEDIR}" -c "${CONFIG}" "${@}"

Here is my output:

All targets have been cleaned
An error was encountered while executing action link
Action link not handled
Action ~/.zsh not handled
Action ~/.vimrc not handled
Action ~/.vim not handled
Action ~/.zshrc not handled
Action ~/.tmux.conf not handled
Installing/updating submodules [git submodule update --init --recursive]
All commands have been executed

==> Some tasks were not executed successfully

This happens no matter what. I have created and destroyed multiple vimrc, zshrc, etc. files, I have completely removed Dotbot and re-downloaded it, I have tried using JSON formatting. Nothing I seem to try will let me create the links. I'm not sure what the problem is at this point; any suggestions will be appreciated.

Phew! Turns out this does a lot better than I had initially planned!

I need help making the test cases, here's the behaviour I tried to create and need to test:

Failure cases

link:
  ~/bin/:
    path: bin
    use_glob: true

defaults:
  - link:
      use_glob: true
link:
  ~/bin/:
    path: bin/

Because we aren't sure if we should link bin inside ~/bin or link bin as ~/bin. Handled with error message by the glob_star_loc is -1 and destination[-1] is '/' conditional.

Success Cases

link:
  ~/bin:
    path: bin
    use_glob: true

defaults:
  - link:
      use_glob: true
link:
  ~/bin:
    path: bin

Works as normal link operation ~/bin -> bin, so that globbing can be turned on at default level.

defaults:
  - link:
      use_glob: true
link:
  ~/bin/:
    path: bin/*

link:
  ~/bin:
    path: bin/*

Links all the items in bin/* into individual links in ~/bin/. So long as there is a wildcard * in the path: then it will link the glob results as individual items into the destination.

Trailing Slash

~/bin vs ~/bin/ is a special case used to enforce a certain behavior, with a trailing slash and use_glob enabled, it forces dotbot to treat the link task as "link items into this folder" instead of potentially linking as.

Advanced Quirks

link:
  ~/.config/tmux/:
    path: tmux/**.conf
    create: true

Links all the .conf files in tmux/ into ~/.config/tmux/ however the double star recursive match is not supported in python's default glob. (but is working in glob2 pip package Example:

tmux
├── 2.1/
│   ├── bind-pane-zoom.conf -> ../bind-pane-zoom.conf
│   ├── keybinds-vi.conf -> ../keybinds-vi.conf
│   ├── mouse-mode-new.conf -> ../mouse-mode-new.conf
│   └── powerline-py3.conf -> ../powerline-py3.conf
├── 2.5/
│   ├── bind-pane-zoom.conf -> ../bind-pane-zoom.conf
│   ├── keybinds-vi.conf -> ../keybinds-vi.conf
│   ├── mouse-mode-new.conf -> ../mouse-mode-new.conf
│   └── powerline-py3.conf -> ../powerline-py3.conf
├── bind-pane-zoom.conf
├── keybinds-vi.conf
├── mouse-mode-new.conf
├── mouse-mode-old.conf
├── powerline-py3.conf
├── py2u.25.tmux.conf
└── tmux.conf

After link operation:

/home/robo/.config/tmux
├── bind-pane-zoom.conf -> /home/robo/code/configs/tmux/bind-pane-zoom.conf
├── keybinds-vi.conf -> /home/robo/code/configs/tmux/keybinds-vi.conf
├── mouse-mode-new.conf -> /home/robo/code/configs/tmux/mouse-mode-new.conf
├── mouse-mode-old.conf -> /home/robo/code/configs/tmux/mouse-mode-old.conf
├── powerline-py3.conf -> /home/robo/code/configs/tmux/powerline-py3.conf
├── py2u.25.tmux.conf -> /home/robo/code/configs/tmux/py2u.25.tmux.conf
└── tmux.conf -> /home/robo/code/configs/tmux/tmux.conf

Notice the folders were not linked. They could have been included as links by simply changing the glob link target to be tmux/* instead of tmux/*.conf.

If glob2 was in use, we could do recursive globs and make the glob match detection better.

I don't think we should bother including glob2 because I think this feature set covers all the use cases we are concerned about, and the cost of inclusion of another library is not worth the benefit here IMO.

Even without glob2, we can still do some interesting and more useful advanced things, like:

link:
  ~/.config/:
    path: config/*/*
    create: true

This example creates normal directories for everything at the ~/.config/* level, but then everything at the ~/.config/*/* level is now a symlink.

Anyways,

I looked at the test setup, but it's a bit too cryptic for me to dive into tonight. Could someone else try making the test cases that match these?

This is my dotbot config YAML.

On line 24 there I try to symlink Sublime Text: ~/Library/Application\ Support/Sublime\ Text\ 3/Packages/User: sublime

The path is correct. However it fails on me when I run my install script.

~/.dotfiles master
❯ ./install
Link /Users/nikivi/.bash_profile -> /Users/nikivi/Dropbox/Mackup/.bash_profile not removed.
Link /Users/nikivi/.gitmodules -> /Users/nikivi/dotfiles/zprezto/.gitmodules not removed.
All targets have been cleaned
Installing submodules [git submodule update --init --recursive]
All commands have been executed
Link exists ~/.zshrc -> /Users/nikivi/.dotfiles/zsh/zshrc.zsh
Link exists ~/.gitconfig -> /Users/nikivi/.dotfiles/git/gitconfig
Link exists ~/.gitignore_global -> /Users/nikivi/.dotfiles/git/gitignore_global
Link exists ~/.hammerspoon -> /Users/nikivi/.dotfiles/hammerspoon
Link exists ~/.ideavimrc -> /Users/nikivi/.dotfiles/idea/ideavimrc
Link exists ~/Library/Application\ Support/Karabiner -> /Users/nikivi/.dotfiles/karabiner
Nonexistent target ~/Library/Application\ Support/Sublime\ Text\ 3/Packages/User -> sublime
Link exists ~/.config/nvim -> /Users/nikivi/.dotfiles/nvim
Link exists ~/Library/Application\ Support/Code/User -> /Users/nikivi/.dotfiles/vscode
Link exists ~/.tmux.conf -> /Users/nikivi/.dotfiles/tmux/tmux.conf
Some links were not successfully set up

==> Some tasks were not executed successfully

I am really not sure why that goes wrong. Shouldn't it work?

Thank you for any help.

Hi,

I want to also use dotbot to management config system-wide instead of only the dotfiles.

Say, for customized or revised stuff under /etc.

How to achieve this purpose?

Setting up Dotbot with an existing dotfiles repository is pretty quick.

It would be cool to have a script to make it easier to set up a dotfiles repository with automatic Dotbot integration. The script could be an interactive program that the user guides in creating a git repository, moving existing dotfiles, and setting up an initial install.conf.yaml automatically based on the files that were moved.

The script should be mostly automated, basically only asking the user for confirmation when doing things.

my current file starts like this:

- defaults:
    link:
        relink: true

- clean: ['~']

- create:
    - ~/.profile.d
    - ~/.local/bin
    - ~/.config/alacritty
    - ~/.config/htop
    - ~/.config/i3status
    - ~/.config/mako
    - ~/.config/redshift/hooks
    - ~/.config/rofi
    - ~/.config/sway
    - ~/.config/swaylock
    - ~/.config/yay
    - ~/.vim/pack/dotfiles

# casual
- link:
    ~/.dotfiles: ''
    ~/.bashrc: bash/bashrc
    ~/.bash_profile: bash/bashrc
...

but it always outputs Action create not handled. So no folders are created. Even the example from the README.md does not work for me.

maybe you could add a better explanation of the yaml configuration, because i do not see the problem.

Hello, I've been using dotbot to manage my dotfiles for some time now. It has worked great thus far, since most applications read from a common configuration directory on all platforms. Sublime Text however, uses different directories depending on what platform you're on.

Would it be feasible to add support for different destination paths depending on your OS? In Python there's sys.platform which contains a string representing the platform Python is running on.

As for configuration syntax, I'm not sure. YAML has a construct where you can map keys to values, see example 2.6, but since dotbot maps links "the other way", the syntax wouldn't make sense (and is syntactically incorrect):

- link:
    {
     linux: ~/.config/sublime-text-3,
     darwin: ~/Library/Application Support/Sublime Text 3,
     win32: ~/Sublime Text 3
    }: Sublime Text 3/

If you flip the target and destination, I think it looks decent and you would still be able to configure links independently of the OS/platform. It would obviously break all existing configuration files, however.

- link:
    Sublime Text 3/:
      linux: ~/.config/sublime-text-3,
      darwin: ~/Library/Application Support/Sublime Text 3,
      win32: ~/Sublime Text 3
    vim/: ~/.vim

What do you think?

Hi. In the YAML configuration file, if you say force: true, does that currently override the desination no matter what it is? I would like it to overwrite any existing symbolic link, but still be safe in all other ways. This is because I keep different versions of files in my dotbot repository, and several YAML files to install different versions, and would like to switch between them on the fly.

I have a further feature request: Possibility to set the default behaviour so that one can limit the verbosity of the YAML file; i.e. set globally force: true or the like.

Thanks for your consideration.

Hello again, @anishathalye :wave:

I recently finished up a small plugin locally for my own use and it's pretty darn useful for a particular use-case that I think might also be useful for a large portion of other users.

This issue is just to gauge your interest in adding it into core (after a thorough code review, of course).

The gist

The plugin is called merge, and it is used like this (subject to changes):

- merge:
    ~/dest: source

As you can see, it works very much like link, however, the difference is that instead of linking dest to source, it "merges" the contents of source into dest (with symlinks dest -> src).

The salient use-case for this is with dotfiles belonging in the $XDG_CONFIG_HOME or ~/.config directory.

Instead of having to do this:

- link:
    ~/.config/foo: config/foo
    ~/.config/bar: config/bar
    ~/.config/baz: config/baz

You instead only have to do this:

- merge:
    ~/.config: config

Ah, much better :smile:

Anyway, I just finished a decent working implementation last night for my own personal use that so far works nicely for my needs.

Let me know if you might be interested in a PR for this. If not, no hard feelings. :+1:

There's actually two requests here, but given the overlap I thought it suitable to just raise a single issue:

1. Introduce a new type of plugin, condition plugins, that provide named, configurable conditions written in Python
2. Introduce a new directive accepting: condition(s) to evaluate and a list of tasks/actions to execute when true

Motivation

There are currently 3 plugins linked from the official documentation that provide a condition and a container of tasks to be executed based on the outcome (if, ifarch, ifplatform). Dotbot users have already adopted this type of plugin so the interest is clearly there.

The most flexible (and similar to this request) is the if plugin which takes a single shell command. Personally, there are at least a couple other conditions that I'd like to share with the community that aren't well suited to a shell one-liner.

Notably, all three of the plugins also share the same issue. They don't propagate defaults from the parent context (and scope any new defaults set to the child context).

I think it would be great to provide improved support for this pattern in the core dotbot project itself.

Examples

First, you can see below a couple options for a hypothetical new directive. I don't actually have a strong preference here. Whatever we think is least likely to conflict with a custom plugin name.

- cond:
    if:
      tty: true
    then:
      - shell:
        - '<command_that_may_prompt_for_input>'
    else:
      - shell:
        - 'fallback_command'

- block:
    when:
      tty: true
    tasks:
      - shell:
        - '<command_that_may_prompt_for_input>'

Next you can see a few variants of the condition. I actually think it makes sense to support all of these.

# For a string, it is interpreted as a shell command (providing consistency with the link directive's current if option) 
- cond:
    if: "command -v ansible"
    then:
      - shell:
        - 'echo "I found ansible in $PATH"'

# For a dict, use each key to find the corresponding Condition plugin and pass the value as arguments
- cond:
    if:
      tty: true
      platform: 'arch'
    then:
      - shell:
        - 'echo "I am running in a TTY on ArchLinux!"'

# An array is also supported for the same reasons as with dotbot tasks/actions.
# It allows repeating a condition multiple times.
- cond:
    if:
      - decrypted: "./config/file/a"
      - decrypted: "./config/file/b"
      - 'command -v "program_requiring_sensitive_config"'
    then:
      - shell:
        - 'echo "I have everything i need to configure <program_requiring_sensitive_config>."'

I'm envisioning the condition plugins following the same pattern as those providing directives. A new test runner class would take the full condition and delegate to the appropriate condition plugins for evaluation.

import dotbot
import sys

class TtyCondition(dotbot.Condition):

    def can_handle(self, directive):
        return "tty" == self._directive

    def handle(self, directive, data=True):
        expected = data if data is not None else True
        return expected == (sys.stdin.isatty() and sys.stdout.isatty() and sys.stderr.isatty())

Other thoughts

1. Unlike #225 the only potential for collision would be the single new directive. Behavior of tasks/actions remains the same as it is today.
2. The if attribute in the current link plugin could also make use of the above conditions. As described above it would be backwards compatible.
3. Why not just implement this as a new plugin? Configuring dotbot/dotbot-plugins as submodules seems to complicate import resolution. It's much easier if the new Condition abstract base class is in the dotbot repository itself. I'm fairly new to python, so maybe there's a pattern I'm missing that would actually make this easier.

what

• Run shell command conditionally (optional)
• Update README docs

how

• the shell directive will check if the 'if' property is available.
  • If it is, it will run the command in 'if' property.
  • If the command results in a return 0 code or 'true', then the 'command' property will run.
  • If the condition to run the command is a non 0 code or 'false', the shell command won't run
• dotbot config example:

- shell:
    - command: echo "this is running on a MacOS"
      if: uname -s | grep -i "Darwin"

• running shell command regardless

- shell:
    - command: echo "This should run regardless"

• dotbot config example: skipping command if false

- shell:
    - command: echo "This command should be skipped"
      if: false

why

• can run the shell command conditionally
• can use multiple dotbot configs to run different OS
  • Ex: incorporating it in https://github.com/ecarlson94/dotbot-template

where

• file changed in ./dotbot/plugins/shell.py

usage

Create dotbot config

- shell:
    - command: echo "this is running on a MacOS"
      if: uname -s | grep -i "Darwin"

- shell:
    - command: echo "This command should be skipped"
      if: false

- shell:
    - command: apt update && apt upgrade -y
      if: lsb_release -i | grep -io 'debian'
      description: Update APT package repository

    - command: dnf update -y
      if: lsb_release -i | grep -io 'fedora'
      description: Update DNF package repository

NOTE

• The reason to wrap the if command in string, is to prevent Python from interpreting it.
  • str(item["if"]),
  • https://github.com/Clumsy-Coder/dotbot/blob/25ef5d5a5fed1e2f95e57115dfdbc8ba67cfc93f/dotbot/plugins/shell.py#L52
  • Try it without wrapping it in strings
    • https://github.com/Clumsy-Coder/dotbot/blob/974156ccdecee05eb9544b0b21cc0351428ddd43/dotbot/plugins/shell.py#L51-L57

    • run_if_result = dotbot.util.shell_command(
          item["if"],
          cwd=self._context.base_directory(),
          enable_stdin=False,
          enable_stdout=False,
          enable_stderr=stderr,
      )
      

    • Fails to run because error was thrown

      - shell:
          - command: echo "This command should be skipped"
            if: false
      

    • Works if wrapping false in strings

      - shell:
          - command: echo "This command should be skipped"
            if: "false"
      

Running d2f76a2, I found that this from the examples:

link:
    ~/.config/:
        glob: true
        path: config/*

... does not work when you only have one file in the config directory.

Example:

$ cd ~/.dotfiles
$ mkdir config
$ touch config/test.ini
$ ./install
...
Ambiguous action requested.
No wildcard in glob, directory use undefined: ~/.config/ -> ['config/test.ini']
Did you want to link the directory or into it?
...

As soon as you add another file, globbing works.

just like LINK directive, there should one for running individual commands for SHELL directive.

Ex:

- shell:
    - command: apt update && apt upgrade -y
      if: lsb_release -i | grep -io 'debian'
      description: Update APT package repository

    - command: dnf update -y
      if: lsb_release -i | grep -io 'fedora'
      description: Update DNF package repository

why

• provides a better flow of running dotbot process
• can contain similar commands in a single shell directive.
  • Ex: shell directive for updating repositories
  • Ex: installing a package using package manager in one OS, downloading a script from another OS

… in which case the destination is assumed to be a folder in which source paths are linked.

This is to solve #301. The diff on github is borked and shows lots of changes, but the changes are actually quite small:

• Link._default_sources was changed to return a list of sources rather than a single source
• path is changed to paths and all of the linking action is wrapped in an additional loop that iterates over source paths
• If globbing is active, additional tests are added for edge cases
• If there is no globbing and several paths are given, the destination path is assumed to be a directory, so we adapt the destination path accordingly

If you're ok with the approach I'll also edit the documentation. I don't really understand how the tests work, but I'll also try to add a few test cases when I have time to look more into it.