-->

Thursday, May 1, 2014

Personal Photo Organization in the Mobile and Cloud Environment

I'm going to describe here a "system" I have developed for organizing my personal photo collection with the intention of making it safe from loss while being easy to use. This type of system probably isn't for everyone, but there may be some useful ideas here.

My particular method relies heavily on the Google Drive product, as well as Android based mobile devices. If this describes the types of products that you are comfortable with, then there is an even higher chance that there are some ideas here that you could use.

The key components of this system are:


Google Drive

Google drive is my choice for cloud storage for the following reasons:

  • Large amount of storage available for free
  • Low prices for getting extended storage
  • Easy to sync contents to a PC
  • Easy to access contents from a mobile device
  • Has extensive productivity apps included for word processing, spreadsheets, and presentations
  • Has open interfaces so that other apps can be connected to it for viewing and editing files within drive
  • Integration into chrome operating system (chromebooks and chromeboxes)

For these reasons, I prefer to store my photos within google drive. You can set up your own google drive at this link: https://drive.google.com/‎

After setting up your google drive, you can create the following folder structure. This is based on organizing your photos into monthly folders.

Photos
  Family
    2013
      2013-01
      2013-02
      etc.
    2014
      2014-01
      2014-02
      etc...

The above represents a folder called Photos with a subfolder called Family. In the Family subfolder are subfolders for each year that you have photos for. Within each year are folders for each month of the year in YYYY-MM format. These folders will contain the actual photos for that month (generally .jpg files). The reason that the year is duplicated in each monthly folder is for those programs which display only the folder name. In those cases you will be able to tell the year and month when just looking at that name.

So this is step 1. At this point you can manually upload your pictures to the correct folder and they will then be stored in the cloud. Or, if you have your photos on a PC and you have set up drive on the PC according to the instructions for google, you can drag the pictures to the correct folder in your PC version of drive and they will get uploaded to google drive automatically.

Photo Acquisition

Manually dragging files into folders can get old pretty quick, there must be an easier way! Actually, there is! The first piece of this is called Google Apps Script. There are ways to write scripts which can access your google drive and automatically move files around. I will introduce at this point a script that I am using to perform this function, as well as some other functions that I will be talking about later.

You can get a copy of the script that I use here: PhotoSizer

At the beginning of the script, you can see a section of code that looks like this:
function run()
{
  move("/Phone/Camera", "/Photos/Family", 0);
  move("/Phone/Camera2", "/Photos/Family", 0);
  move("/Docs/DougAndChris/chrisphone/camera", "/Photos/Family", 0);
  move("/NewPhotos", "/Photos/Family", 0);
  sync("/Photos/Family/2014", "/PhotosResized1024/Family/2014", 1600);
}
The key sections are the move functions. Here, I am specifying that all file in /Phone/Camera (the Camera subdirectory of the Phone directory) should be moved into /Photos/Family into the correct year and month subfolders according to when the photo was taken. The photo will also be renamed to the following format YYYYMMDDHHMMSS-<OriginalName>, where the YYYYMMDDHHMMSS are according to the information in the photo's exif information for when it was taken. This is so that the photos sorted according to name in the folder will be in the same order in which they were taken. You can copy this script into your own google drive, change it so that it works with folders into which you will dump your photos, run the "run" function once to set up the authorizations, and then set a trigger such that the "run" function runs every 15 minutes or so. At this point, comment out the sync function (I'll get to that later).

Once this is set up, you can set up various methods to dump pictures into these folders, and then they will get copied into the correct area of your google drive. At a minimum, you should be able to manually drag files on your PC to one of the folders that you have set up to initially receive your photos.

Here are the two methods that I use for my photo acquisitions.

Phone Camera

Getting photos from your phones camera to your google drive is fairly simple to do by using the share function and uploading the photo to the correct incoming directory on your google drive. However, I prefer something a bit more automated. This is where the android program "FolderSync" comes in. I can set it up to sync a certain folder on my phone (ie. the camera roll folder) with a certain folder on my google drive (ie. one of the directories I set up on my google drive to contain my phone camera photos). I can set up this sync to occur on a certain schedule, whenever there is a new photo, or when I manually start it - depending on my desires for data usage on my phone).

FolderSync is available here: FolderSync

Here is a view of the settings that I am currently using for the sync of my camera to google drive:






Whenever I take a picture on my phone, it gets uploaded to google drive - and then the script I have set up to run will shortly move the photo to the correct folder. Since I am syncing deletions, the photo on my phone will eventually get deleted from my phone when a sync occurs at a later time.

Camera

Most cameras do not have wifi yet, nor the ability to synchronize their contents with a google drive folder. There are a couple of solutions here. Since I am in the process of going without a pc, the solution that I am currently using involves an android phone (or tablet).

Step 1 is to get the photo from the camera's memory card to the phone. For that, I am using a Kingston MobileLite. Some of the newer phones support USB OTG (On The Go), which allows you to plug an adapter and a standard memory card reader into the phone in order to transfer the files off the camera memory card into the phone. If you have a phone which supports this option, it would probably be a cheaper and easier way to perform this transfer (This is available on amazon at http://www.amazon.com/gp/product/B00DKMUBXQ/). I have actually transitioned to using the USB OTG reader, since my newest phone now supports this option.

Once the photos are in the phone, I can use the process described above (FolderSync) for transferring the photos to google drive. However, before performing this transfer, I suggest that you ensure the rotations are correct on the photos. You can use my android application PhotoRoto to fix up the rotation of the photos so that they'll display with the correct rotation on google drive, or any other program that you intend to use to display these photos.

Speaking of fixing photo rotation - some phone cameras also have problems with putting out jpg files which can be oriented correctly regardless of the program viewing the photo. I have an LG Optimus G, which does not require any fixing. However, my wife has an LG G2, which does have a problem in this area. If you are using a phone which has problems with rotations, you can use PhotoRoto to fix those as well prior to uploading them to google drive. In that case, you would probably want to change the synchronization on those phones camera so that it is manual only rather than automatic on a schedule or after a photo is taken so that you have the opportunity to fix the photos before they are uploaded.

Photo Consumption

Photo consumption refers to how you will enjoy your photos once you have taken them. There are some definite advantages to google drive in this respect. First of all, it is very easy to just use a standard browser and access your photos in google drive on the web. Google drive will display thumbnails of your photos if you view a folder, can bring up large sized previews with the ability to click to the next and previous photo, and you can easily download photos to the computer in case you need to do something like edit the photo or send it in an email.

If you have google drive synchronization to a PC, then the photos will show up on your PC on the google drive folder, and you can use all your favorite PC tools to organize, edit, or view your photos.

Google drive also has an android application which is very nice for viewing the folders and photos right within google drive with thumbnails, large previews, scrolling back and forth, and the ability to download the photo if needed.

Photo Resizing

The above may be all that you need, but my experience is that I want to be able to access my photos quickly on my tablet or phone without having to rely on a fast data connection to do so. In order to do that, I need the photos loaded on my phone. However, I have over 120GB  (30,000) of photos, and I don't have that amount of storage available on my phone to store photos. In order to solve that problem, I have resized all of my photos to a smaller format which is just fine for viewing on a phone or sending in an email or sharing on facebook. All of my 120+ GB of photos fit within 4GB of memory on my phone when using these resized versions.

Remember my google apps script called photosizer which had a role in moving photos from incoming directories in google drive to the correct place with the correct name for some decent organization?

You can get a copy of the script that I use here: PhotoSizer

At the beginning of the script, you can see a section of code that looks like this:
function run()
{
  move("/Phone/Camera", "/Photos/Family", 0);
  move("/Phone/Camera2", "/Photos/Family", 0);
  move("/Docs/DougAndChris/chrisphone/camera", "/Photos/Family", 0);
  move("/NewPhotos", "/Photos/Family", 0);
  sync("/Photos/Family/2014", "/PhotosResized1024/Family/2014", 1600);
}
I said that I would talk about that function called sync, and that you should comment it out. This is where you can now change that line to appropriate paths in your google drive. In the above, I am synchronizing any photos in /Photos/Family/2014 to the path /PhotosResized1024/Family/2014. The third parameter is the max edge size you want to use on your resized photos. In this case, I am using 1600 pixels (despite the name of the dir where I implied that I am using 1024 lol). When this script runs, any photo that shows up in any any subdir or /Photos/Family/2014 which be copied into the corresponding subdirectory in /PhotosResized1024/Family/2014 except that it will be resized to a maximum dimension of 1600 pixels.

There are a couple of things that you need to do to make this script happy.
  • None of the photos or any directory in the path can have a space in the name
  • The /Photos/Family directory and all subdirectories need to have privacy changed to public
Since google apps script engine runs on google servers, there are some restrictions that you need to be aware of. Primarily, a script is only allowed to run for 5 or 6 minutes, and if it runs any longer it is killed. It takes quite awhile to scan directories for photos that need to be resized, and even longer to perform a single resize. I restrict my regularly scheduled run to only go through a years worth of pictures, and I change this every year, in order to limit how long it takes to run so that it actually can complete. If you have a lot of photos that need to be resized at once, it may only get a portion of them done before it times out. If this happens, it's not too much of a problem. It will run every fifteen minutes (or however often you have set it up to run) and will eventually get through them all.

Once you have resized photos available on google drive, it's a simple matter of setting up another folder pair in FolderSync on your phone to synchronize the resized photos on google drive with a folder on your phone or tablet's memory card. You can set this to run on a regular basis (nightly, or hourly). If you have a lot of photos, you may want to set this to only synchronize the current year (by specifying the correct folder pairs), so that it doesn't take an excessive amount of time to run.

I find that a great program for quickly viewing a large amount of photos on your android phone is QuickPic. In my case, I have about 30,000 photos which are nicely handled by QuickPic.

Conclusion

I'd be interested in hearing your ideas on how to organize photos efficiently. If you have any questions or comments on the above process, please leave a comment below. I did not go into too many details on many of the items above, but instead focused on more of an overview. Your feedback will help me understand what I may need to explain further.
Print this post

1 comment:

  1. I created a web application to perform the photo rotations on the back end of the process (when the google drive appscript is run), rather than by processing the photos via photoroto. This considerably simplifies the work flow process described above. Take a look here for more information on that webapp: http://dougchartrand.blogspot.com/2016/05/developing-google-app-engine-web-service.html

    ReplyDelete