Best software to extract all photos from one hard drive and transfer to another HD

ederek · Oct 16, 2008

bmattock said:
I will gladly send a copy of my script to anyone who wants it. I do not know if it will work on Macs, but it should run on both Unix/Linux and Windows PC's IF you have Perl and Exiftool installed (both are freeware). You should probably know a bit of Perl scripting too, I wrote my scripts for me, you may wish to change things.

Bill - a copy of your scripts would be most appreciated. I like your workflow (and rigorous backups!). Haven't used ExifTool before (only Exifer, a simple program for stripping exif data prior to posting, for example).

Can you clarify one of the features of ExifTool listed on Phil Harvey's site?? It is this one that stumps me: "Decodes a riddle wrapped in a mystery inside an enigma". I can only imagine the wonderful enigmas and mysteries to be found in my ole pics, but am not very good at figuring out riddles, so it may be a futile effort...😀

The features look very good. Now that I'm also shooting with an RD1, will start carrying the GPS logger around again, and also the Olympus voice recorder (have Dragon for processing audio data). The tool would be nice for adjusting errors in the camera's time (or when I travel and forget to change timezones!), versus using the built-in time-shift feature within the geocoding software.

Going back to film with the M-4's, I realize there's a LOT I miss about Digital. But I also like the gear, manual shooting process and results. Need a process for handling all the scans, and would be interested in how you tag your scans.

shadowfox · Oct 16, 2008

rsync works for me. It allows me to resume copying should I need to stop it for some reason (long mirroring session).

On Linux and Mac OS X, it came with the OS. On Windows, install Cygwin.

It's probably a bit too computer-geeky to set these up but I just want to throw this in as an option. 🙂

kuzano · Oct 16, 2008

Another sort of "geek" option....

Another sort of "geek" option....

Operating on the assumption that hard drives are cheap... (relatively so), I often "clone" the hard drive in my computer.

There is a software publisher by the name of Acronis. They publish a disk management and imaging software called Truimage (now in version 11 if I recall correctly).

One of the many functions of Truimage is to allow you to "clone" every software aspect of the hard drive in your computer to another hard drive. It's commonly used when you need to switch to a bigger hard drive and don't want to sit for hours re-loading the O/S, all your programs, and then transferring data. Since it creates a mirror image, you can clone from a 120 GB HD to a 500GB HD, for instance.

My favorite use of the program is to clone a new drive, whether I need a bigger drive or not. After the cloning process, I put the new hard drive on the shelf for protection against the possibility of a hard drive crash.

Now, If I put the new hard drive in the EXACT computer it came from, I can just swap it in (about five minutes), boot the system and be up an running. (Down side-this is not a regular backup as it is in the state when the "clone" process took place, although you can clone on a regular basis if you want). But, you should still be doing consistent and regular backups. My trick can save me a weekend of rebuilding my hard drive from scratch.

There are other similar clone programs, like Norton Ghost. I am most familiar with Acronis.

I work on computers and before I take on a particularly risky job with regard to potential data loss in doing the job, I commonly clone the drive in the customer computer. It takes about a half hour. This safety feature has saved my bacon numerous times.

So, you can clone your drive and put the copy on the shelf for ten minute disaster recovery. If you no longer have the SAME EXACT computer you cloned FROM, you can hook the drive up to a new computer as a second drive and roam or gather the contents of the "cloned" drive, just as if it were an auxiliary drive, or an external drive.

Acronis Truimage is for Windows computers.. All Windows O/S.

Don't know about Mac/Apple, or Linux. I do drive a Saab and wear Birkenstocks, but never got around to buying a MAC or Apple. Also, there is not enough geek time in the world to deal with Linux (Although I've loaded Red Hat, Denebian and Ubuntu over the years and just don't see the justifying factors considering the tradeoffs). Windows may have some issues from time to time, but it is the mainstream in computing and affords me a great opportunity to make a reasonable living.

bmattock · Oct 17, 2008

ederek said:
Bill - a copy of your scripts would be most appreciated. I like your workflow (and rigorous backups!). Haven't used ExifTool before (only Exifer, a simple program for stripping exif data prior to posting, for example).

Can you clarify one of the features of ExifTool listed on Phil Harvey's site?? It is this one that stumps me: "Decodes a riddle wrapped in a mystery inside an enigma". I can only imagine the wonderful enigmas and mysteries to be found in my ole pics, but am not very good at figuring out riddles, so it may be a futile effort...😀

The features look very good. Now that I'm also shooting with an RD1, will start carrying the GPS logger around again, and also the Olympus voice recorder (have Dragon for processing audio data). The tool would be nice for adjusting errors in the camera's time (or when I travel and forget to change timezones!), versus using the built-in time-shift feature within the geocoding software.

Going back to film with the M-4's, I realize there's a LOT I miss about Digital. But I also like the gear, manual shooting process and results. Need a process for handling all the scans, and would be interested in how you tag your scans.

File enclosed - if you can read Perl, I'm sure you'll have no trouble with it. Looking it over, I see I used the UNIX command 'utime' to update the time/date stamp of the newly-copied file. This of course won't work on other OS's. However, you can use the 'FileModifyDate' is a 'fake' ExifTool attribute - if you modify it and then invoke the 'WriteInfo' routine (currently commented out), it will do the same thing, and should work on any OS.

I'm not doing much in the way of Exif data modification in this script - really just reorganizing files based on the digital photo's time/date stamp if it has one ('CreateDate'). However, you can see my commented sections - you can modify any attribute you wish and when you invoke the 'WriteInfo' method after that, everything will be written.

Give it a try - on some nice safe data, of course, and make backups first, etc. You know the deal. But your data should be left just where it was - only copies are made.

You need Perl and the Find, Copy, ExifTool and Time::Local Perl modules - all freely downloadable if you don't already have them. They're all on CPAN except for ExifTool.

If you need some help, let me know.

Code:

#!/usr/bin/perl -w
#
# This script is designed to find image files and move them to a specified
# directory structure, based on their file date/time stamp.
#

use strict;
use File::Find;
use File::Copy;
use Image::ExifTool;
use Time::Local;

my $exifTool = new Image::ExifTool;

my $scanned_image_loc = "/home/bmattock/storage/Oldest_Recovered_Photos/";
my $final_image_loc   = "/media/disk/Oldest_Recovered_Photos/";
my @scanned_images;
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst);
my $mtime;
my $tempstring;
my $filename;
my $touchname;
my $doit_flag="YES";

find(\&wanted, $scanned_image_loc);

# Get the time/date stamp for scanned images
foreach(@scanned_images)
{
	my $success = $exifTool->ImageInfo($_);
	my $val = $exifTool->GetValue('CreateDate');
	if(!defined($val))
	{
#		print "No CreateDate Attribute for $_\n";
		$doit_flag="NO";
	}
	else
	{
#		print "Yes, there is a CreateDate Attribute for $_\n";
		$doit_flag="YES";
                $year=substr($val,0,4);
		if($year eq "0000"){$doit_flag="NO";}
                $mon=substr($val,5,2);
                $mday=substr($val,8,2);
                $hour=substr($val,11,2);
                $min=substr($val,14,2);
                $sec=substr($val,17,2);
	}
	if($doit_flag eq "YES")
	{
#		print "YEAR: $year MONTH: $mon DAY: $mday\n";
		$tempstring=$final_image_loc.$year;
		if (!-d $tempstring){mkdir $tempstring;}
		$tempstring=$tempstring."/".$mon;
		if (!-d $tempstring){mkdir $tempstring;}
		$tempstring=$tempstring."/".$mday;
		if (!-d $tempstring){mkdir $tempstring;}
		$filename=substr($_,rindex($_,"/"));
		$touchname=$tempstring.$filename;
		if(!-e $touchname)
		{
			print "Copying file:$_ to: $tempstring\n";
#			exit;
			copy($_,$tempstring) or die "Copy $_ to $tempstring failed: $!";
			my $info = $exifTool->ImageInfo($_);
			my $copyright="Copyright 2008, Your Name Here, All Rights Reserved";
#			$exifTool->SetNewValue(Copyright => $copyright);
#			$exifTool->WriteInfo($touchname);
#			Set timestamps to be the same as original files
#			print "Setting timestamp\n";
			$mtime = timelocal($sec,$min,$hour,$mday,$mon-1,$year);
			utime $mtime, $mtime, $touchname or die "Error setting timestamp for $touchname: $!n";
		}
		else
		{
			print "Not copying file:$_ to: $tempstring because it exists already.\n";
		}
	}

}


sub wanted 
{
        if(/.jpg$|.JPG$|.tif$|.TIF$|.bmp$|.BMP$|.pef$|.PEF$|.XCF$|.xcf$|.PSD$|.psd$/)
        {
#		print "File: $_\n";
		push(@scanned_images,$File::Find::name);
	}
}

bmattock · Oct 17, 2008

ederek said:
Can you clarify one of the features of ExifTool listed on Phil Harvey's site?? It is this one that stumps me: "Decodes a riddle wrapped in a mystery inside an enigma". I can only imagine the wonderful enigmas and mysteries to be found in my ole pics, but am not very good at figuring out riddles, so it may be a futile effort...😀

I guess what he means is that there are no standards for Exif that EVERYONE uses. Most agree on MOST of the Exif fields, but some have their own proprietary stuff, and some pack data, or encrypt it, and they don't release their method, so people have to reverse-engineer it - if they can.

However, most of the attributes that he's decoded are terrific and work for everything. If you try to write to or read from an attribute that your particular file type doesn't support, it just won't, and you'll know it.

Fun fact - ExifTool will also read/write IPTC and XMP fields - in MP3 files, PDF, video, etc. Even some document file types. How cool is that? I've organized my old expense reports (Excel) and MP3 files the same way as my photos. Different top-level directory of course.

bmattock · Oct 17, 2008

ederek said:
The features look very good. Now that I'm also shooting with an RD1, will start carrying the GPS logger around again, and also the Olympus voice recorder (have Dragon for processing audio data). The tool would be nice for adjusting errors in the camera's time (or when I travel and forget to change timezones!), versus using the built-in time-shift feature within the geocoding software.

I cheat and use the Linux program digiKam for my GPS data integration into my photo files. My datalogger is the Wintec WBT200, a cheap unit that works well - I use gbsbabel to download the data into an XML file, then simply tell digiKam to merge the file with my selected photos.

Going back to film with the M-4's, I realize there's a LOT I miss about Digital. But I also like the gear, manual shooting process and results. Need a process for handling all the scans, and would be interested in how you tag your scans.

I added the GPS data with my script manually, after having obtained it from a similar shot I took with a digital camera. Sometimes I'll take a shot with a compact digital just to get a time/date stamp for my later film processed and scanned. Makes it easy to match up. Still a bit manual and awkward, though.

I scan my negatives with either my Minolta DualScan IV or my Epson 4490 (medium format). I then use a script similar to my previous upload, except that I grab the time/date stamp from FileModifyDate or my own time/date stamp if I took the photos on some previous date instead of from 'CreateDate'. I write that value to DateTimeOriginal and CreateDate, along with any other fields I find useful. Here's a good example:

Code:

bmattock@slimline:~$ exiftool -s ./Photos/vuescans/Scan-080920-0038.jpg
ExifToolVersion                 : 7.00
FileName                        : Scan-080920-0038.jpg
Directory                       : ./Photos/vuescans
FileSize                        : 5 MB
FileModifyDate                  : 2008:09:20 18:26:25
FileType                        : JPEG
MIMEType                        : image/jpeg
JFIFVersion                     : 1.1
ResolutionUnit                  : inches
XResolution                     : 300
YResolution                     : 300
ExifByteOrder                   : Little-endian (Intel)
ImageDescription                : Farmer's Market
Make                            : Minolta
Model                           : Scan Dual IV
PageName                        : Transparency
Copyright                       : Bill Mattocks
DateTimeOriginal                : 2008:09:20 12:00:00
CreateDate                      : 2008:09:20 18:25:10
GPSVersionID                    : 2.0.0.0
GPSLatitudeRef                  : North
GPSLongitudeRef                 : West
GPSAltitudeRef                  : Above Sea Level
GPSAltitude                     : 220 m
GPSMapDatum                     : WGS-84
ImageWidth                      : 4499
ImageHeight                     : 2984
EncodingProcess                 : Baseline DCT, Huffman coding
BitsPerSample                   : 8
ColorComponents                 : 3
YCbCrSubSampling                : YCbCr4:2:0 (2 2)
GPSLatitude                     : 42 deg 39' 11.16" N
GPSLongitude                    : 83 deg 19' 59.52" W
GPSPosition                     : 42 deg 39' 11.16" N, 83 deg 19' 59.52" W
ImageSize                       : 4499x2984

Then I can use my previous script to move the file based on my having created / updated the 'CreateDate' attribute.

Best software to extract all photos from one hard drive and transfer to another HD

ederek

Well-known

shadowfox

Darkroom printing lives

kuzano

Veteran

bmattock

Veteran

bmattock

Veteran

bmattock

Veteran