Dealing with Google Takeout in Mac OS


After July 2019, Google Photos will no longer sync with Google Drive. Whether or not this is a good thing depends on specific use cases. Regardless, it is happening and this impacts the way that I use Photos. Generally when I take photos on my Pixel 3 phone, I just let the system handle things. The phone syncs with Google Photos in the cloud. Google Photos syncs with Google Drive and Apple’s Time Machine backs up my Google Drive onto a local disk drive. I have been comfortable with this. I am not a heavy curator of my photos. I have some albums in Google Photos but generally I want my photos organized by creation date. This is important because the files that Google syncs to my Google Drive do not include the album folders or meta data. I’m ok with this.

Some people have noted with dismay that deleting Photos on your phone can cause propagating deletes throughout the synchronized stores causing a cherished photo to disappear. If you do not back up the Google Drive like I do, you are out of luck. If you run out of space on your phone you may have no other option but to delete a bunch of photos. My pixel 3 has no SD card so I will hit this point some day.

After Google stops synchronization between Google Photos and Google Drive, the pictures I keep in Google Photos will become distinct from the ones I keep in Google Drive. The phone will continue to sync with Google Photos but new pictures will not make their way into Google Drive (and my Time Machine backup).

So how do I back up my collection of pictures? One way I have found is to use Google Takeout to download a set of zip files containing all of my pictures along with Google’s own metadata, fomatted as JSON files.

Goggle Takout was designed for the use case of someone who needs to get all of their stuff out of the Google Cloud. It’s generic enough to handle all of the data formats that Google has developed over the years. This includes Google Photos. While the original intent of the Takeout service may have been to leave the Google-sphere, the service only creates a copy of the data. It does not remove it. This makes it suitable for backup purposes.

The Takeout service is found by going to the My Account page and selecting Data and Personalization -> Download Your Data. After you marvel at all the data Google does keep, the easiest path is click the Deselect All option, then scroll down to Google Photos and select that. After that click the Next Step button and customize the download options.

When I did this I wound up with 10 zip files. Takeout limits each zip file to two GB and the files within each zip are selected based on what will fit within the limit rather than using a strictly logical organization. From there you are on your own.

Troubles Using Finder

At first I assumed that I simply needed to unzip the files and move them to a common folder. However the files are organized to fill up two GB files efficiently. This means many folders have been duplicated across the zip files. In other words the files in any particular folder may have been spread across several zip files. A given folder name may exist in multiple zip files each with a subset of the total set of files in that folder. Mac’s Finder does not deal with this in the way that I expected.

I assumed that I could drag and drop folders to a common destination folder and that a proper merge would occur. Instead Finder prompts me with a dialog that offers two options: Stop the move, or Replace the contents of the common destination folder with the contents of the source folder. Selecting Replace deletes the current contents before moving the new contents. This is not a merge.

After some internet searching I found a hidden option that performs a merge. Holding the Option key while performing the drag and drop adds a Merge option to the the dialog box that pops up when a name collision occurs. This sounds like what I want but it turns out that is is not my idea of a merge. This operation will select the newest version of an item and delete the older one. This includes folders not just files. Since the folder names are duplicated across these zip files, this so-called merge is no better than a strict replacment operation (other than the date selection). I still wind up with many files getting lost.

Using Ditto to Merge Google Takeout Zip Files

Back to the internet. I found out that Mac OS has a ditto command which performs a proper recursive merge of a file hierarchy. Furthermore, it can take multiple zip files as the source and let you skip the unpacking step.

I copied the zip files into a work folder and issued the following command:

ditto -x -k -V \ \ \ \ \ \ \ \ \ \
	result 1>log.txt 2>&1

This created the target folder result and merged all of the files into their proper folders. No files were lost in this process. Now I copy these files to my local backup disk. I will have to repeat this periodically. Unfortunately there is no incremental takeout feature so I will have to repeat the full download. Then I can use the rsync command to update the backup. I’ll leave that for another day.

17 thoughts on “Dealing with Google Takeout in Mac OS

  1. Would this tool be a good process for using takeout to get a copy of all my photos from the google ecosphere organized properly for upload into the Apple ecosystem? I downloaded my take-outs and have (3) large zip folders but as you mentioned with the meta data etc it is very ill organized

    1. I have a Pixel phone and I have not used Apple’s cloud to manage my photos. So I don’t know what “organized properly” means in the Apple context. However, using the “ditto” command with Takeout files will organize the files in folders by date, or album name. All of the metadata is retained in JSON files in each folder. So if you have added descriptions and location information it will be in those JSON files. Potentially an importer program could be written to use this information to create albums in another system.

  2. I have downloaded the approx. 950GB that I have been storing in Google Drive (I had a rash of bad drive issues a year ago) in the form of multiple 50GB .tgz files. I don’t have enough space on my SSD to store and unpack these files. Can I run this operation from my external 4TB drive on which these files currently reside?

  3. questions from a Terminal beginner: I’m wanting to use this process for my music instead of photos. Google Play Music is transferring to YouTube music, and its an inferior app that doesn’t do what google did. I copied your commands, but got the error messages: Unknown compression type and Couldn’t read PKZip signature. Why? What should I be doing differently?

  4. I am not good with terminal coding. I don’t understand how to get this working, can you help with step by step instructions please?

  5. Wow! Thank you for this code and write-up. Saved me headache and time and trouble. I tried so many other things but this just worked.

  6. Hey, I usually never comment on stuff but this little command saved my friggin life! I downloaded about 800GB of photos from google takeout and getting all of that sorted was a logistical nightmare. I just used a wildcard in my command to make it a bit more concise. All of my takeout zip files were from google photos and moved into a working directory like was suggested.

    ditto -x -k -V takeout-*.zip result 1>log.txt 2>&1
    then in a separate console tab I ran
    tail -f log.txt
    so I could watch what was happening in real time.

Leave a Reply

Your email address will not be published. Required fields are marked *