Best software to extract all photos from one hard drive and transfer to another HD

[eleskin]

My M8 has been generating thousands of photos and this is the problem:
I was wondering what would be the best software that could identify every photo in one hard drive, and woud then transfer all the photos in order of date to the new hard drive. I have many photos in all kinds of files, and would like to have a drive dedicated only to my photo archive and have all the photos in one giant file in the order in which the photos were taken.

 

[mervynyan]

windows explorer?

 

[bobbyrab]

As far as I know your computer will only list items by filename, but if you have photoshop bridge you could view the files by date created, then rename them once you have them in that order, then the computer will view them in the right order, you would have to do this one folder at a time but keep continuous numbering from folder to folder, then drag the folders to the new drive.

 

[bobbyrab]

I should have added you could do the same thing with all the images in one folder if your computer could cope with size of the folder, I've 5gb of ram, but lightroom for example will get a bit cranky if I have too many files in the one folder, bridge might be the same.

 

[ederek]

eleskin - what OS are you using?

If on Windows or Linux, I highly recommend "Beyond Compare" by Scooter Software. The latest version has a new feature allowing you to compare using a Flattened Folder Structure - a powerful new feature. There is a demo here:
http://www.scootersoftware.com/videos/WN3FlattenAndAlign.html

I use Beyond Compare to back up new files to my backup drive (identifies and syncs only new files). Also useful for putting in a Memory Card which contains previously downloaded images, plus new captured images (e.g. you didn't empty the card after transferring last time...) - result is that you can quickly transfer over the new files onto the hard drive (or Move).

Be cautious if you are using Lightroom. I think in that case you may want to move the original files from within lightroom, so the edits overlaying the untouched original are carried with the file (may be lost and require a re-import if moved outside the application, not 100% on this).

 

[bmattock]

My M8 has been generating thousands of photos and this is the problem:
I was wondering what would be the best software that could identify every photo in one hard drive, and woud then transfer all the photos in order of date to the new hard drive. I have many photos in all kinds of files, and would like to have a drive dedicated only to my photo archive and have all the photos in one giant file in the order in which the photos were taken.

I have written a number of Perl scripts to do this.

I use Perl, which is a free scripting language, and Exiftool, which is a freeware tool that reads and optionally writes Exif meta data in a variety of file, including image files of all sorts (jpeg, tiff, all flavors of RAW, etc).

My scripts do this.

1) Find all files meeting a minimal profile in a specified directory. For example, all the '*.jpg' files in the C:\Photos directory (I use Linux, but am assuming you mean Windows, it would work on either one).

2) Read the names of all the files into an array.

3) Walk through the array, extracting the date-time stamp from each photo.

4) Create a repository as follows (as an example):

C:\Date_Organized_Photos <- Base Repository
C:\Date_Organized_Photos\2008 <- Year
C:\Date_Organized_Photos\2008\02 <- Month
C:\Date_Organized_Photos\2008\02\05 <- Day

My script looks to see if the required directory exists; if it does not, it creates it. Then it copies the file into that directory.

5) Change the date-stamp on the newly-copied file to be the same as the internal Exif date-time stamp, for consistency.

That's it. You end up with a file tree that is easy to parse and search, based on dates. It is also useful, I have found, when your camera's numbering scheme rolls over (mine goes to 10,000, then starts again) so you do not accidentally overwrite one file with another of the same name.

I have added on lots of other features over the years, to do things you probably don't need, like adding Exif data for scanned negatives instead of digital photos, GPS meta-data, and Exif, XMP and IPTC meta-data describing the photograph for future use in searching (for example, all photos which include a tree, or all photos taken in Kenosha, Wisconsin, as opposed to trying to remember when I took a given photo).

I will gladly send a copy of my script to anyone who wants it. I do not know if it will work on Macs, but it should run on both Unix/Linux and Windows PC's IF you have Perl and Exiftool installed (both are freeware). You should probably know a bit of Perl scripting too, I wrote my scripts for me, you may wish to change things.

Just drop me a line if you want it. No charge.

I am currently archiving digital files dating back to 1998 (I was an early adopter) and film-based files that I've scanned, going back to 1973. All organized as I've described. Working on reducing duplicates created by poor handling of files, I'm down to about 36,000 files and about 300 gigs of photo files (I scan to TIFF and keep maximum sizes). I run my scripts in an automated fashion nightly to first one, then a second, external USB drive. Therefore, I have three copies of each file (at least, I also have stacks of old, obsolete hard drives I no longer use, but keep around). I also regularly transfer one of my external drives to another location and swap it with the one I keep there - so one fairly current backup is always offsite in case of disaster.

 

[Nh3]

http://www.faststone.org/FSViewerDetail.htm

 

[Richard Marks]

If you have this many images in a year or 2 the long haul is a bit daunting. Film is looking good from this perspective!

Richard

 

[bmattock]

If you have this many images in a year or 2 the long haul is a bit daunting. Film is looking good from this perspective!

Richard

Then shoot film.

 

[Quinn Porter]

Downloader Pro is a great tool for downloading and would work perfectly for the task you describe.

 

[Richard Marks]

Then shoot film.

Thank you I do
Actually a combination of both film and digital suits me fine.

Richard

 

[oftheherd]

I have written a number of Perl scripts to do this.

I use Perl, which is a free scripting language, and Exiftool, which is a freeware tool that reads and optionally writes Exif meta data in a variety of file, including image files of all sorts (jpeg, tiff, all flavors of RAW, etc).

My scripts do this.

1) Find all files meeting a minimal profile in a specified directory. For example, all the '*.jpg' files in the C:\Photos directory (I use Linux, but am assuming you mean Windows, it would work on either one).

2) Read the names of all the files into an array.

3) Walk through the array, extracting the date-time stamp from each photo.

4) Create a repository as follows (as an example):

C:\Date_Organized_Photos <- Base Repository
C:\Date_Organized_Photos\2008 <- Year
C:\Date_Organized_Photos\2008\02 <- Month
C:\Date_Organized_Photos\2008\02\05 <- Day

My script looks to see if the required directory exists; if it does not, it creates it. Then it copies the file into that directory.

5) Change the date-stamp on the newly-copied file to be the same as the internal Exif date-time stamp, for consistency.

That's it. You end up with a file tree that is easy to parse and search, based on dates. It is also useful, I have found, when your camera's numbering scheme rolls over (mine goes to 10,000, then starts again) so you do not accidentally overwrite one file with another of the same name.

I have added on lots of other features over the years, to do things you probably don't need, like adding Exif data for scanned negatives instead of digital photos, GPS meta-data, and Exif, XMP and IPTC meta-data describing the photograph for future use in searching (for example, all photos which include a tree, or all photos taken in Kenosha, Wisconsin, as opposed to trying to remember when I took a given photo).

I will gladly send a copy of my script to anyone who wants it. I do not know if it will work on Macs, but it should run on both Unix/Linux and Windows PC's IF you have Perl and Exiftool installed (both are freeware). You should probably know a bit of Perl scripting too, I wrote my scripts for me, you may wish to change things.

Just drop me a line if you want it. No charge.

I am currently archiving digital files dating back to 1998 (I was an early adopter) and film-based files that I've scanned, going back to 1973. All organized as I've described. Working on reducing duplicates created by poor handling of files, I'm down to about 36,000 files and about 300 gigs of photo files (I scan to TIFF and keep maximum sizes). I run my scripts in an automated fashion nightly to first one, then a second, external USB drive. Therefore, I have three copies of each file (at least, I also have stacks of old, obsolete hard drives I no longer use, but keep around). I also regularly transfer one of my external drives to another location and swap it with the one I keep there - so one fairly current backup is always offsite in case of disaster.

bmattock - good to see you posting again. You've been missed. Sounds like a useful script you have there. I never messed with Perl, although I often told myself I should.

eleskin - Windows explorer will do what you want the first time around for sure. Just expect to let you computer work a while. Bring up the explorer. Go to your drive(s) with all your folders with photos. Hold down the control key and click on each one. Right click on one and drag all to the drive you want them in, release the button and select copy all or move all (suggest copy to avoid possible problems with corrupted files).

Or: Do a search for all *.jpg files (left click on start). When it is done, click on the first, go to the bottom, hold down the shift key, and click on the last one. Click the minimize window in the upper right (the middle one), right click on any of the files, holding down the button, and move all to the drive you want them in and release the button. Select either copy all or move all as you wish (I suggest copy in case of corruption). You will have all the files there. Unsorted by date, type, subject or anything else useful.

Or: contact bmattock for his program. 😀

Sorry for the sillyness. I just needed to unwind from some tedious work at the office today.

 

[williams473]

You shouldn't need any extra software if you're on Windows. MAC - I don't know, but with Windows XP or Vista, just do a file search and in the "filename" field type *.jpg or *.tiff or whatever file type you want. The "*" symbol tells Windows you want all filenames with the specified file extension. Make sure you also tell the search agent to look on "My Computer," which will check all drives in their entirety. I assume you haven't "hidden" any files - if so then you will know how to unhide them, which you don't want to forget to do before initiating the search.

Once the search finds all the files, just select all, copy, and paste into a folder on your new drive, assuming both drives are on the same system. With that many large files it's going to take a while. Good luck.

 

[bmattock]

bmattock - good to see you posting again. You've been missed. Sounds like a useful script you have there. I never messed with Perl, although I often told myself I should.

Technically, you could use any scripting software, or even a compiled language, and accomplish the same thing. I use Perl because I know how to - when all you have is a hammer, everything looks like a nail. It's pretty painless, actually. And thanks for the kind words.

eleskin - Windows explorer will do what you want the first time around for sure. Just expect to let you computer work a while. Bring up the explorer. Go to your drive(s) with all your folders with photos. Hold down the control key and click on each one. Right click on one and drag all to the drive you want them in, release the button and select copy all or move all (suggest copy to avoid possible problems with corrupted files).

Or: Do a search for all *.jpg files (right click on start). When it is done, click on the first, go to the bottom, hold down the shift key, and click on the last one. Click the minimize window in the upper right (the middle one), right click on any of the files, holding down the button, and move all to the drive you want them in and release the button. Select either copy all or move all as you wish (I suggest copy in case of corruption). You will have all the files there. Unsorted by date, type, subject or anything else useful.

The biggest problem with that method is one I found out myself, the hard way. Duplicate file names overwrite each other or do not copy. The former is disastrous, the latter gives false security.

Or: contact bmattock for his program. 😀

Sorry for the sillyness. I just needed to unwind from some tedious work at the office today.

Me too. Although I am writing Perl, so I should be very happy.

 

[bmattock]

You shouldn't need any extra software if you're on Windows. MAC - I don't know, but with Windows XP or Vista, just do a file search and in the "filename" field type *.jpg or *.tiff or whatever file type you want. The "*" symbol tells Windows you want all filenames with the specified file extension. Make sure you also tell the search agent to look on "My Computer," which will check all drives in their entirety. I assume you haven't "hidden" any files - if so then you will know how to unhide them, which you don't want to forget to do before initiating the search.

Once the search finds all the files, just select all, copy, and paste into a folder on your new drive, assuming both drives are on the same system. With that many large files it's going to take a while. Good luck.

Again, just a word of warning. If there are duplicate file names, the duplicates will either be overwritten, or ignored and not copied. It can be troublesome. If there are no duplicates, then yes, this would work fine.

However, it only copies the files into one big honking directory, not sorted by date. I mean, the files may well be sorted by date within the directory, but they are not in subdirectories according to date.

 

[bottley1]

try copy and paste......then go and make a coffee! 😛

 

[SixPM]

Lightroom is my ideal choice, you could move all the images from one drive to another and you can sort and view the images by date.

 

[sjw617]

A USB cord between both.... Right click, send a copy, choose the c drive on the other computer? Generally pictures tend to be placed in My Pictures. OR just copy the My Documents folder (all the good stuff to back up).
OR
Copy files to a thumb drive and transfer them multiple files at a time?

Steve

 

[bmattock]

Lightroom is my ideal choice, you could move all the images from one drive to another and you can sort and view the images by date.

It doesn't store them that way, it just sorts them that way. It's not the same thing.

 

[bmattock]

A USB cord between both.... Right click, send a copy, choose the c drive on the other computer? Generally pictures tend to be placed in My Pictures. OR just copy the My Documents folder (all the good stuff to back up).
OR
Copy files to a thumb drive and transfer them multiple files at a time?

Steve

Great for backup, not for selecting files of particular interest (like image files only) and backing up. As to a thumb drive - I don't know about you, but my image files total 300+ gig. You got any thumb drives that size? I shoot more than the typical thumb drive will hold in a weekend.