Tools for writing awesome Fabric files

Do run() instead of run_as_root() so user can decide context using a with statement such as:

with fabtools.python.virtualenv('/home/example/python'): fabtools.require.python.distribute()

For some reasons (gremlins ?) the test (res == "1") does not work even if the psql command returns "1" I found it worked better using ("1" in res), without any str.strip(), etc...

For some reason sudo('getent passwd name') dont work for me. So I've found this solution (work @bearstech and on my own debian)

I've also improved user creation to allow using more parameters like the essential --disabled-password

I also think you should add a way to copy some ssh keys to the newly created account. maybe user.create(..., authorized_keys=fd)

This is a realy common task

Hello, i've got some issues using fabtools, could you please help me to solve it:

[[email protected]] sudo: sed -i.bak -r -e '/en_US.UTF-8\ UTF-8/ s/^([[:space:]]*)#[[:space:]]?/\1/g' "$(echo /etc/locale.gen)" [[email protected]] out: sudo: export: command not found [[email protected]] out:

Fatal error: sudo() received nonzero return code 1 while executing!

Requested: sed -i.bak -r -e '/en_US.UTF-8\ UTF-8/ s/^([[:space:]])#[[:space:]]?/\1/g' "$(echo /etc/locale.gen)" Executed: sudo -S -p 'sudo password:' export PATH="$PATH:"/srv/sites/reeliner"" && sed -i.bak -r -e '/en_US.UTF-8\ UTF-8/ s/^([[:space:]])#[[:space:]]?/\1/g' "$(echo /etc/locale.gen)"

Aborting. Disconnecting from [email protected]... done.

here's the code I try: require.postgres.database(env.database_name, env.user)

Update is_running to actual return whether or not a service is running. The status command succeedes no matter if the service is running or not. This has is_running check for the presence of 'running' in the output of service <service> status.

service.is_running currently just checks to see whether the call to service <service> status succeeds. This is not enough to determine whether the service is running or not.

This PR:

• fixes already added PPA detection (path was wrong for the sources.d)
• add normalization of the repo for the PPA check (Ubuntu >= 12.04 replaces dots with underscores)
• adds auto_accept=True to pass to add-apt-repository if on Ubuntu >= 12.04
• adds the possibility to specify a keyserver

In details:

Normalization now works on 10.04 and 12.04 so I closed #70 and #71.

As for 10.04, not sure how we want to handle the confirmation, as there is no option like you mentioned before.

For now, if on ubuntu >= 12.04, it confirms by default.

For the key server, I made sure that if they tried to pass multiple key servers, to just grab the first one (instead of erroring), but it may not be needed.

Feedback welcome, thank you !

Hi Ronnix,

Here is a little suggestion/starter to manage repository. It offers basic functions to clone, update or reset a repositiory.

Here are some remarks/questions:

• When error occurs, like Control Version System is not supported, it just prints a red message. Would you prefer that it raises an exception ?
• It deals only with Git repositories.
• How can I write Fabtools tests ? Actual test files are not very explicit.

Let me know if you're ok with this implementation. It's very basic but it should be enough for most of the usages (unless you don't use Git).

Hi ronnix,

Since the Fabric 1.5.0 release, Fabtools OpenVZ commands don't work anymore. It complains about not finding the quiet import from fabric.api . I'm ok to make a pull request for that but I don't know where to start. Could you point me what to do to solve this problem ?

Whatever thanks for fabtools which helps us a lot.

Frank

Added SmartOS support for fabric.

pip uninstall -y fabtools ; pip install git+git://github.com/scalp42/fabtools.git@smartos

Feel free to spin up a box on Joyent and test using the following snippet:

env.hosts = ['joyent_box']

def provision():
    pkg.update_index(force=True)
    pkg.upgrade(full=True)
    pkg.install('top')
    if pkg.is_installed('top'):
            print(green('top is installed'))
    if not pkg.is_installed('redis'):
            print(red('redis is not installed'))
    pkg.install('unzip', update=True, options=['-V'])
    pkg.uninstall(['unzip', 'top'], orphan=True)
    if pkg.smartos_build().startswith('joyent'):
            print('SmartOS Joyent')
    print(cyan(pkg.smartos_image()))

UPDATE: added support for passing yes string for packages that require license validation like sun-jre6 on SmartOS for example.

require.pkg.package('sun-jre6', yes='yes')

When I use this fonction fabtools.require.system.locale('fr_FR.UTF-8')

I get this error message

[[email protected]:2222] sudo: echo 'fr_FR.UTF-8 UTF-8' >> /var/lib/locales/supported.d/local [[email protected]:2222] out: /bin/bash: /var/lib/locales/supported.d/local: No such file or directory

Fatal error: sudo() received nonzero return code 1 while executing!

Requested: echo 'fr_FR.UTF-8 UTF-8' >> /var/lib/locales/supported.d/local Executed: sudo -S -p 'sudo password:' /bin/bash -l -c "echo 'fr_FR.UTF-8 UTF-8' >> /var/lib/locales/supported.d/local"

Aborting.

Warning: md5sum: /var/lib/locales/supported.d/local: No such file or directory

My full script is here http://pastebin.com/aVPW41Xb

Fabric version 1.4.3 Fabtools version 0.7.0 Vagrant version 1.0.3 Virtualbox image Debian Squeeze AMD 64 https://s3-eu-west-1.amazonaws.com/rosstimson-vagrant-boxes/Debian-Squeeze-64-rvm.box

currently fabtools installs the nginx package, which doesn't contain useful modules like secure_link, this patch allows you to specify an alternate nginx package if you so desire without it being removed by fabtools

i running the unittest of fabtools like

FABTOOLS_TEST_REUSE_VM=1 FABTOOLS_TEST_BOX="ubuntu/xenial64" pytest -x

platform linux2 -- Python 2.7.14, pytest-3.7.4, py-1.6.0, pluggy-0.7.1

rootdir: /home/zodman/dev/fabtools, inifile:
collected 191 items                                                                                                                                                                                               

fabtools/tests/test_apache.py .........                                                                                                                                                                     [  4%]
fabtools/tests/test_deb.py .                                                                                                                                                                                [  5%]
fabtools/tests/test_files.py F

==================================================================================================== FAILURES =====================================================================================================
_______________________________________________________________________________________ FilesTestCase.test_default_temp_dir _______________________________________________________________________________________

self = <fabtools.tests.test_files.FilesTestCase testMethod=test_default_temp_dir>, is_file = <MagicMock name='is_file' id='140068732546704'>, md5sum = <MagicMock name='md5sum' id='140068732332624'>
put = <MagicMock name='put' id='140068732345808'>, umask = <MagicMock name='umask' id='140068732391824'>, owner = <MagicMock name='_owner' id='140068732413200'>
mode = <MagicMock name='_mode' id='140068732438672'>

    def test_default_temp_dir(self, is_file, md5sum, put, umask, owner, mode):
        owner.return_value = 'root'
        umask.return_value = '0002'
        mode.return_value = '0664'
        from fabtools import require
        from fabric.api import env
>       require.file('/var/tmp/foo', source=__file__, use_sudo=True)

fabtools/tests/test_files.py:64: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
fabtools/require/files.py:201: in file
    func('chmod %(mode)o "%(path)s"' % locals())
fabtools/utils.py:24: in run_as_root
    return func(command, *args, **kwargs)
../../.virtualenvs/fabtools/local/lib/python2.7/site-packages/Fabric-1.14.0-py2.7.egg/fabric/network.py:684: in host_prompting_wrapper
    host_string = raw_input("No hosts found. Please specify (single)"
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_pytest.capture.DontReadFromInput object at 0x7f644c686610>, args = ()

    def read(self, *args):
>       raise IOError("reading from stdin while output is captured")
E       IOError: reading from stdin while output is captured

../../.virtualenvs/fabtools/local/lib/python2.7/site-packages/_pytest/capture.py:647: IOError
---------------------------------------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------------------------------------
No hosts found. Please specify (single) host string for connection: 

and ever show the error:

No hosts found. Please specify (single) host string for connection:

I read the code and the error its at: test_files.py what not had interaction with vagrant like (functional_tests).

how fix it ? move the test_files.py to functional_test/ to get working with vagrant ?¿

i had the feeling test_files.py its a incomplete unitttest.

i had:

fabtools.user.add_ssh_public_key("web", "id_rsa_deploy.pub")

traceback:

sudo: touch "/export/web/.ssh/authorized_keys"
Traceback (most recent call last):
  File "/home/zodman/.virtualenvs/provision/local/lib/python2.7/site-packages/fabric/main.py", line 757, in main
    *args, **kwargs
  File "/home/zodman/.virtualenvs/provision/local/lib/python2.7/site-packages/fabric/tasks.py", line 386, in execute
    multiprocessing
  File "/home/zodman/.virtualenvs/provision/local/lib/python2.7/site-packages/fabric/tasks.py", line 276, in _execute
    return task.run(*args, **kwargs)
  File "/home/zodman/.virtualenvs/provision/local/lib/python2.7/site-packages/fabric/tasks.py", line 173, in run
    return self.wrapped(*args, **kwargs)
  File "/home/zodman/work/sysadmin_interalia/provision/fabfile.py", line 47, in install
    fabtools.user.add_ssh_public_key("web", [ "id_rsa_deploy.pub",])
  File "/home/zodman/.virtualenvs/provision/src/fabtools/fabtools/user.py", line 236, in add_ssh_public_key
    add_ssh_public_keys(name, [filename])
  File "/home/zodman/.virtualenvs/provision/src/fabtools/fabtools/user.py", line 267, in add_ssh_public_keys
    use_sudo=True)
  File "/home/zodman/.virtualenvs/provision/src/fabtools/fabtools/require/files.py", line 201, in file
    func('chmod %(mode)o "%(path)s"' % locals())
TypeError: %o format: a number is required, not str

posible fix on

diff --git a/fabtools/require/files.py b/fabtools/require/files.py
index e140d7b..79b91ea 100644
--- a/fabtools/require/files.py
+++ b/fabtools/require/files.py
@@ -198,6 +198,8 @@ def file(path=None, contents=None, source=None, url=None, md5=None,
         mode = 0o666 & ~int(umask(use_sudo=True), base=8)
 
     if mode and _mode(path, use_sudo) != mode:
+        if isinstance(mode, str):
+            mode = int(mode,base=8)
         func('chmod %(mode)o "%(path)s"' % locals())
 

Up until the latest release, mode argument in fabtools.require.files.file was used as it came, whether it was an integer or a string.

In the current repository version, this behavior has changed and now an integer in base 8 is required.

In my opinion, this change, a part from being a problem for not being backwards compatible, is dangerous and counter intuitive, since if anyone forgets about the 0o bit, the file gets wrong permissions silently.

As an example, using mode=600 leads to an ugly ---x-wx--T permissions string as if mode=384 had been used.

I would like this to be reverted to interpreting any integer as base 10 or, at least, continue accepting strings as input.

https://github.com/fabtools/fabtools/blob/4c0be62ec09b9d8dfb3a3995b3f4010c21b069d9/fabtools/files.py#L95

fabtools.files.mode function uses the stats command passing the path as a string (with quotes):

'stat -c %%a "%(path)s"'

Somehow this prevents bash to expand special characters such as the tile or the asterisk, so this call fails:

fabtools.files.mode('~/.ssh/id_rsa')

This doesn't happen if the double quotes are removed from the stats command:

'stat -c %%a %(path)s'