Repair broken bookmarks to referenced files in Apple Photos

Overview

Repair Apple Photos Bookmarks

Work in progress to repair file location bookmarks in Apple Photos.

Background

Starting in macOS 10.15/Catalina, photos imported into the Apple Photos app and not copied into the library (e.g. referenced photos) are tied to a specific volume on a specific computer. If you move the photos to a different volume or computer, even with the same name, Photos will be unable to open the original files and will produce a "Missing file" dialog. This is caused because of macOS Sandbox security features and the fact that Photos now stores locations to photos using security-scoped file system bookmarks instead by path. This script will repair your photos library by updating the bookmarks.

This is a work in progress and it's a bit of hack. It requires a few manual steps to get it working (the script will prompt you).

How to use

pip install -r requirements.txt python3 repair_photos_bookmarks.py follow the prompts

Additional Information

For more information, see this discussion

Comments
  • The photos need to be in the same path?

    The photos need to be in the same path?

    Sorry if this is a stupid question, but I'm confused by this statement in the readme file:

    the absolute path to the photos must remain the same. For example, if your photos were located on an external drive named "Fotos" and in a folder named "MyFotos" the path would /Volumes/Fotos/MyFotos, the new volume must also be named "Fotos" and the folder must be named "MyFotos".

    But isn't the whole point of the script to solve the problem that

    If you move the photos to a different volume or computer, even with the same name, Photos will be unable to open the original files and will produce a "Missing file" dialog.

    So I must be misunderstanding something when I think: the problem is that Photos no longer sees my files because the path has changed but this can only be fixed if the files are in the same path...

    Could you explain what I'm missing?

    opened by tophee 3
  • Add ability to reconstruct bookmark from ZFILESYSTEMBOOKMARK.ZPATHRELATIVETOVOLUME and ZFILESYSTEMVOLUME.ZNAME when the ZBOOKMARKDATA blob is NULL

    Add ability to reconstruct bookmark from ZFILESYSTEMBOOKMARK.ZPATHRELATIVETOVOLUME and ZFILESYSTEMVOLUME.ZNAME when the ZBOOKMARKDATA blob is NULL

    Thanks, @RhetTbull. No joy with that, unfortunately, but I've been able to run it fine on a non-M1 Mac.

    Although I've since discovered it doesn't quite help for my use case. I have a Photos library that was originally an iPhoto library with half the photos were referenced and while I still have the folder of images, the link to them is now broken as they're no longer on the same volume. But it seems that this script reads and updates the ZBOOKMARKDATA blob and I was hoping it would be able to write missing ZBOOKMARKDATA blobs from the path data, as because this library came from iPhoto there is no existing ZBOOKMARKDATA data.

    But don't worry, I'll explore further options. Thanks for your time on this script though!

    Originally posted by @purplespider in https://github.com/RhetTbull/RepairPhotosBookmarks/issues/3#issuecomment-1140998968

    enhancement 
    opened by RhetTbull 0
  • psutil.NoSuchProcess: process no longer exists (error on M1 Mac)

    psutil.NoSuchProcess: process no longer exists (error on M1 Mac)

    Struggling to get past this error when running:

    WARNING: this program will rewrite your Photos library database and could result in total data loss.
    Please ensure you have a backup and that the Photos app is not running!
    Do you want to proceed? [y/N]: y
    Please open Photos and uncheck the box 'Importing: Copy items to the Photos library' in Photos Preferences.
    Type 'y' when you have done this. [y/N]: y
    Please quit Photos.
    Type 'y' when you have done this. [y/N]: y
    Traceback (most recent call last):
      File "/opt/homebrew/lib/python3.9/site-packages/psutil/_psosx.py", line 343, in wrapper
        return fun(self, *args, **kwargs)
      File "/opt/homebrew/lib/python3.9/site-packages/psutil/_psosx.py", line 401, in cmdline
        return cext.proc_cmdline(self.pid)
    ProcessLookupError: [Errno 3] assume no such process (originated from sysctl(KERN_PROCARGS2) -> EINVAL)
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/Users/james/Downloads/RepairPhotosBookmarks-main/repair_photos_bookmarks.py", line 356, in <module>
        main()
      File "/opt/homebrew/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
        return self.main(*args, **kwargs)
      File "/opt/homebrew/lib/python3.9/site-packages/click/core.py", line 1055, in main
        rv = self.invoke(ctx)
      File "/opt/homebrew/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/opt/homebrew/lib/python3.9/site-packages/click/core.py", line 760, in invoke
        return __callback(*args, **kwargs)
      File "/Users/james/Downloads/RepairPhotosBookmarks-main/repair_photos_bookmarks.py", line 249, in main
        while photos_is_running():
      File "/Users/james/Downloads/RepairPhotosBookmarks-main/repair_photos_bookmarks.py", line 46, in photos_is_running
        return any(p.name() == "Photos" for p in psutil.process_iter())
      File "/Users/james/Downloads/RepairPhotosBookmarks-main/repair_photos_bookmarks.py", line 46, in <genexpr>
        return any(p.name() == "Photos" for p in psutil.process_iter())
      File "/opt/homebrew/lib/python3.9/site-packages/psutil/__init__.py", line 628, in name
        cmdline = self.cmdline()
      File "/opt/homebrew/lib/python3.9/site-packages/psutil/__init__.py", line 681, in cmdline
        return self._proc.cmdline()
      File "/opt/homebrew/lib/python3.9/site-packages/psutil/_psosx.py", line 348, in wrapper
        raise NoSuchProcess(self.pid, self._name)
    psutil.NoSuchProcess: process no longer exists (pid=27506)
    

    I'm on an M1 Mac Studio. Any pointers appreciated.

    opened by purplespider 10
  • Current implementation doesn't update ZINTERNALRESOURCE.ZFILESYSTEMVOLUME info

    Current implementation doesn't update ZINTERNALRESOURCE.ZFILESYSTEMVOLUME info

    Although the current implementation appears to work -- as in Photos is able to locate the photos after repairing bookmarks -- it does not update the volume info in ZINTERNALRESOURCES.ZFILESYSTEMVOLUME. ZINTERNALRESOURCES.ZFILESYSTEMBOOKMARK points to the now updated bookmark, which appears to be correct, but if the volume changed, the volume info in the ZFILESYSTEMVOLUME table pointed at by ZINTERNALRESOURCES.ZFILESYSTEMVOLUME will be incorrect.

    Should the script also add a new volume (if necessary) to ZFILESYSTEMVOLUME and update the pointer in ZINTERNALRESOURCES? Nothing appears to be broken but it does leave an inconsistency in the database.

    opened by RhetTbull 3
  • Allow remapping of old path to new path when updating bookmarks

    Allow remapping of old path to new path when updating bookmarks

    The current implementation uses CFURLCreateByResolvingBookmarkData which fails if the target file isn't present. This means Photos bookmarks cannot be migrated to a different path (unless copies of the target photos were present in both the old location and the new location). If the bookmark resolution is replaced with mac_alias calls, the URL could be extracted even if the target isn't present and then changed, according to some mapping, to the new location.

    Would still need a way to map old location to new location.

    The following python code shows how to extract the bookmark URL.

    """Extract file path from a CFURL bookmark without calling Objective C"""
    import os
    
    from mac_alias import Bookmark, kBookmarkPath
    
    
    def get_bookmark_path(bookmark_data: bytes) -> str:
        """Get the path from a CFURL file bookmark
        This works without calling CFURLCreateByResolvingBookmarkData
        which fails if the target file does not exist
        """
        try:
            bookmark = Bookmark.from_bytes(bookmark_data)
        except Exception as e:
            raise ValueError(f"Invalid bookmark: {e}")
        path_components = bookmark.get(kBookmarkPath, None)
        if not path_components:
            return None
        return "/" + os.path.join(*path_components)
    
    
    if __name__ == "__main__":
        import sys
    
        bookmark_file = sys.argv[1]
        with open(bookmark_file, "rb") as f:
            bookmark_data = f.read()
        bookmark_path = get_bookmark_path(bookmark_data)
        print(f"{bookmark_path=}")
    
    opened by RhetTbull 4
Owner
Rhet Turnbull
I tinker with various things. Here are a few of my projects.
Rhet Turnbull
Gallery written in Python to manage your photos

GalleryMan Gallery written in Python to manage your photos Installation

Asian Cat 24 Dec 18, 2022
Fix datetime EXIF data in photos downloaded from Google Takeout

fix-google-takeout Warning Use at your own risk. Backup your photos. Overview Google takeout for photos

Mayank Mandava 20 Nov 5, 2022
A tool to maintain an archive/mirror of your Google Photos library for backup purposes.

Google Photos Archiver Updated Instructions 8/9/2021 Version 2.0.6 Instructions: Download the script (exe or python script listed below) Follow the in

Nick Dawson 116 Jan 3, 2023
Convert photos to paintings with python

Convert-photos-to-paintings Before the changes After the changes Before the changes After the changes This code is written in the Python programming l

Amir Hussein Sharifnezhad 3 May 31, 2022
Program designed to mass edit and watermark all photos in a directory

Photographer-All-In-One This is a program designed for photographers to mass edit or watermark photos (.jpg || .png) You can run this program from any

Brad Martin 2 Nov 23, 2021
Generates images of calendar month tables and can paste them onto suitable photos.

?? calendizer README Generates images of calendar month tables and can paste them onto suitable photos. A quick way to make your own calendar for prin

Sean Ryan 2 Dec 14, 2022
Convert HDR photos taken by iPhone 12 (or later) to regular HDR images

heif-hdrgainmap-decode Convert HDR photos taken by iPhone 12 (or later) to regular HDR images. Installation First, make sure you have the following pa

Star Brilliant 5 Nov 13, 2022
Find target hash collisions for Apple's NeuralHash perceptual hash function.💣

neural-hash-collider Find target hash collisions for Apple's NeuralHash perceptual hash function. For example, starting from a picture of this cat, we

Anish Athalye 630 Jan 1, 2023
Simple AI app that is guessing color of apple in picture

Apple Color Determinant Application that is guessing color of apple from image Install Pillow, sklearn and numpy, using command for your package manag

Gleb Nikitin 1 Oct 25, 2021
A Python Script to convert Normal PNG Image to Apple iDOT PNG Image.

idot-png-encoder A Python Script to convert Normal PNG Image to Apple iDOT PNG Image (Multi-threaded Decoding PNG). Usage idotpngencoder.py -i <inputf

Lrdcq 2 Feb 17, 2022
Viewer for NFO files

NFO Viewer NFO Viewer is a simple viewer for NFO files, which are "ASCII" art in the CP437 codepage. The advantages of using NFO Viewer instead of a t

Osmo Salomaa 114 Dec 29, 2022
An esoteric visual language that takes image files as input based on a multi-tape turing machine, designed for compatibility with C.

vizh An esoteric visual language that takes image files as input based on a multi-tape turing machine, designed for compatibility with C. Overview Her

Sy Brand 228 Dec 17, 2022
Paper backup of files using QR codes

Generate paper backups for Linux. Currently command-linux Linux only. Takes any file, and outputs a "paper backup": a printable black-and-white pdf fu

Zachary Vance 27 Dec 28, 2022
Easy to use Python module to extract Exif metadata from digital image files.

Easy to use Python module to extract Exif metadata from digital image files.

ianaré sévi 719 Jan 5, 2023
HTML2Image is a lightweight Python package that acts as a wrapper around the headless mode of existing web browsers to generate images from URLs and from HTML+CSS strings or files.

A package acting as a wrapper around the headless mode of existing web browsers to generate images from URLs and from HTML+CSS strings or files.

null 176 Jan 1, 2023
Small wrapper around 3dmol.js and html2canvas for creating self-contained HTML files that display a 3D molecular representation.

Description Small wrapper around 3dmol.js and html2canvas for creating self-contained HTML files that display a 3D molecular representation. Double cl

David Meijer 1 Dec 2, 2021
A minimal, standalone viewer for 3D animations stored as stop-motion sequences of individual .obj mesh files.

ObjSequenceViewer V0.5 A minimal, standalone viewer for 3D animations stored as stop-motion sequences of individual .obj mesh files. Installation: pip

csmailis 2 Aug 4, 2022
Pyconvert is a python script that you can use to convert image files to another image format! (eg. PNG to ICO)

Pyconvert is a python script that you can use to convert image files to another image format! (eg. PNG to ICO)

null 1 Jan 16, 2022