This enables users to pass any option supported by tempfile.NamedTemporaryFile() to atomic_write(). However, mode, dir, and delete continue to be overwritten as they were previously, to enforce the atomic write behavior.

Let me know if this approach is alright. I'm happy to adjust as needed.

Fixes #37

So, I've been playing with the MyPy static type checker lately, and I decided spend some downtime adding type annotations to Python projects I've worked on.

I had to change the logic a bit in order to make the code type check successfully, mainly by more eagerly converting a "unicode or bytes" path to unambiguously unicode when the object is constructed.

I also made it so that tox -e py-typecheck will run the type checker.

I'm using this lib on Windows 10 with no issues so far, but I'm affraid os.rename, though not well documented for Windows, isn't atomic. As per the documentation:

If you want cross-platform overwriting of the destination, use replace(). https://docs.python.org/3/library/os.html#os.rename

So I suggest something along the lines of:

def _replace_atomic(src, dst):
        try:
                os.replace(src, dst)
        except:
                os.rename(src, dst)
        _sync_directory(os.path.normpath(os.path.dirname(dst)))

Ill submit a pull request and I would appreciate if you could update your pip entry. Thanks

Primitive approach implemented here is to fix outstanding temporary files on Windows - it is especially noticeable for long-running processes temporarily saving some state every couple minutes.

The changes in #38 broke the ability to use prefix, suffix and bufsize arguments when creating the tempfile. This pull request restores the ability, and is intended to fix #39

Sometimes I have a bunch of operations which create many files in a folder. The folder should only be there in the end of the operation. Something like this:

with atomic_folder("foo") as folder:
    unzip(something, folder)
    # "foo/" doesn't exist yet.
# now it does

On POSIX it's possible to fsync the directory, to ensure that the new file names are written to disk. I think on Windows it's necessary to open the new (target) file and fsync it.

I think this is missing

if hasattr(fcntl, "F_FULLFSYNC"):
    fcntl.fcntl(f.fileno(), fcntl.F_FULLFSYNC)
else:
    os.fsync(f.fileno())

on OSX.

As mentioned here [0] and here [1] fsync doesn't wait for the changes to hit the disk there.

[0] https://lists.apple.com/archives/darwin-dev/2005/Feb/msg00072.html [1] https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/fsync.2.html#//apple_ref/doc/man/2/fsync

Fun fact: flake was set to run with python 2.7 and 3.5, but 3.5 was not in the build matrix.

Also did some simplification, since we could remove some conditionals, variable, etc.

I'm talking specifically about this code:

        except:
            try:
                self.rollback(f)
            except Exception:
                pass
            raise

I managed to hit a corner case where rollback throws an exception (f was None at that point), so control flow reached the raise statement on the last line. For some reason, this raised the inner exception, not the outer one.

I can submit a patch to fix this: would you prefer a fix that uses the three-argument raise statement, which only works with Python 2, or should I add a dependency on six, and then call six.reraise, so it works with both 2 and 3?

Would it be possible to support passing in pathlib.Path for the path atomic_write(path)?

I noticed that doing so works fine for PosixPaths, but fails for WindowsPaths from this CI run. This can result in someone breaking cross-platform support without being obvious. I notice that the test follows the same solution I went with where you just cast the path to a str.

Allowing atomic_write to accept Paths also helps atomic_write behave more like open as well which also accepts Paths.

I haven't delved too far, but the fix might be as simple as

try:
    from pathlib import Path
    if isinstance(path, Path):
        path = str(path)
except ImportError:
    # Just assume `path` is already a `str`
    pass

pypi just told me i had to enable 2fa to keep uploading this package. because I thought that was an annoying and entitled move in order to guarantee SOC2 compliance for a handful of companies (at the expense of my free time), i deleted the package and published a new version, just to see if the warning disappears. it did, so that's great.

what i didn't consider is that this would delete old versions. those are apparently now gone and yet it's apparently not possible for me to re-upload them. i don't think that's sensible behavior by pypi, but either way i'm sorry about that. the API has been the same since the first release anyway.

I have a python script that outputs file(s) to a specific directory and may be running on multiple computers within a cluster. The output files should all be named like name.#.csv where # is an increasing number: 1, 2, 3, etc. I don't want any files to be overwritten, but I also don't want the write to fail. Instead, it should just increment the file number until the write succeeds. I originally used a dumb loop when writing the file to find the next available file to use, but I'd run into race conditions with many processes running simultaneously and files would somewhat frequently get overwritten.

My solution was the edit I've added here as PR. Basically it adds a "path generator" that should be a generator which sequentially returns the next filename to try. This is flexible, as the user could make the generator return paths based on something entirely different than a sequential numbering. With this, my script works great and I haven't seen anymore files getting overwritten, so I thought I'd add a PR in case others find it useful. If not, feel free to just close this!

Here is an example of how I use it:

import itertools
from atomicwrites import AtomicWriter

def file_incrementor(base_filename):
    for i in itertools.count(1):
        # change i to 1 to see error raised by library to avoid infinite loop
        yield base_filename.format(i)

writer = AtomicWriter(path='name.{}.csv', path_generator=file_incrementor)
with writer.open() as output:
    # calculations with intermittent writes to output
    output.write('example text')
# it also saves the actual path that was saved (regardless of whether path_generator was used)
print(writer.final_path)

When I overwrite a file, the permissions of the file are changed:

$ touch foo

$ chmod 0123 foo

$ stat foo | sed -n 's/^\(Access.*\)Uid.*$/\1/p'
Access: (0123/---x-w--wx)  

$ python3
Python 3.7.3rc1 (default, Mar 13 2019, 11:01:15) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from atomicwrites import atomic_write
>>> with atomic_write('foo', overwrite=True) as f:
...  f.write('foo')
... 
3
>>> 

$ stat foo | sed -n 's/^\(Access.*\)Uid.*$/\1/p'
Access: (0600/-rw-------)  

The normal non-atomic method of overwriting a file does not change the mode:

$ touch foo

$ chmod 0777 foo

$ stat foo | sed -n 's/^\(Access.*\)Uid.*$/\1/p'
Access: (0777/-rwxrwxrwx)  

$ python3
Python 3.7.3rc1 (default, Mar 13 2019, 11:01:15) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> with open('foo', 'w') as f:
...  f.write('foo')
... 
3
>>> 

$ stat foo | sed -n 's/^\(Access.*\)Uid.*$/\1/p'
Access: (0777/-rwxrwxrwx)  

It would be nice to have an option to preserve the mode of the original file.

Unsure if anybody needs this. I might need this because in one usecase I'm writing a lot of files (to different filenames), and only need a guarantee that a SIGKILL won't leave a partially written file (at the target location, tmpfiles are irrelevant).

Also this might be a problem with SSDs, as mentioned in #6