Internal HD organization & enterprise grade?

[icebear]

OMG ... these pc issues will get me grey hair.

I love photography, I can find my way around in LR but organizing my data in an efficient
manner is just a headache.

Current problem (Lenovo Think Station, Win7, 1TB HD int., 2TB ext.) :
My hard drive is maxed out, only 11GB free space BUT if I roughly count all individual folders with program
files and the Library folders (documents, music, video, photo) I get only about 470GB. Most likely a windows
backup on the same disc (stupid anyway) has doubled the data.
I guess I've confirmed some message that I should have denied. BUT: I can't find/see these backups in the explorer. :bang:


Q 1: Does anyone have an advice how to locate the data filling up the drive?
I did empty the trash😉 already and cleaned the disc in utilities which only got me a couple of lousy MB's.

Having this trouble, I was already looking into an additional internal HD and of course since Leica
doesn't built hard drives 😀, there are too many choices.

I narrowed it down pretty much to SATA III, 4TB, 64MB cache, 7200rpm but consumer ratings at the
usual sources are all over the place with about 20% crappy ratings for all manufacturers.

I came across HGST Hitachi (a WD sub division/company) and their "enterprise grade" Ultrastar range
(5 year warranty!), similar Seagate's "Constellation" or WD's own Black "RE". All of these are supposedly
built for longer life and permanent use, up to 2 Mio. hrs until failure.

Q 2: Does anyone have experience with consumer grade vs enterprise grade HD's ?

Thanks in advance for some 101 help.

 

[GaryLH]

I am a mac user.. Haven't used windows in ages. With that being said

The backup data u can't c, I suspect is a hidden partition on your HD. Maybe a diagnostic mode boot disk will spot it. I know a Linux boot disk w/ the use of fdisk from cmd line should spot a hidden partition as well.

Also try going into view options and activate show hidden folders.. U may spot folders that are not being cleaned as they should.

I never store my photo or multimedia folders on my computer even when it is a desktop like my iMac. I always use external HD. That way I never have to worry about the limited space of the internal HD getting full.

Depending on how important your data is depends on what type of backup strategy u use or the type of external storage. But it is not a good thing to have your backup area on the same HD as your your normal.

Whether consumer grade or enterprise, u can still have a HD failure w/ in the same timeframe. However the odds of it happening to a enterprise drive is much much smaller due to difference in design goals. Normally a consumer grade drive is manufactured to a very specific cost goal, whereas an enterprise is designed w/ specific goals more related to reliability such as a mean time to failure, better heat dissipation type design, more read/write heads under data for better throughput..,etc. usually will notan find a mtf number on a consumer grade drive..

Good luck
Gary

 

[bwcolor]

I use enterprise drives in redundant raid configuration as my backup. Enterprise drives are more reliable than consumer drives and work well as a backup drive. I have two raid network servers and one directly connected. Two can undergo the loss of one drive and one can undergo the loss of two drives and maintain data. My working drives are also raid, but are setup for speed. My system drives are Raid O solid state drives and my working drive is a four drive raid 0 external drive. So, these drives are unprotected and more subject to failure, but provide very fast IO, whereas the backup drives are setup as relatively slow, but with great redundancy. I use TimeMachine to backup to the local raid drive and Crashplan to backup to the other two drives and offsite.

 

[Jockos]

Q 1: Does anyone have an advice how to locate the data filling up the drive?

Check the disk manager, maybe the disc space is not partitioned.

Q 2: Does anyone have experience with consumer grade vs enterprise grade HD's ?

Get two consumer grade and run in RAID, rather than relying on a single enterprise drive; all drives will fail one day, enterprise or not. The drive quantity is far more important than quality for your needs, since you don't require 99.999% up time.

 

[Cyrus]

Get a Mac 🙂

 

[craygc]

I use enterprise drives in redundant raid configuration as my backup. Enterprise drives are more reliable than consumer drives and work well as a backup drive. I have two raid network servers and one directly connected. Two can undergo the loss of one drive and one can undergo the loss of two drives and maintain data. My working drives are also raid, but are setup for speed. My system drives are Raid O solid state drives and my working drive is a four drive raid 0 external drive. So, these drives are unprotected and more subject to failure, but provide very fast IO, whereas the backup drives are setup as relatively slow, but with great redundancy. I use TimeMachine to backup to the local raid drive and Crashplan to backup to the other two drives and offsite.

I know this is digressing from the original topic but for your own backup strategy you also need to ask questions beyond "how do I recover if a drive fails?". As a backup strategy, using RAID volumes, you also need to ask what happens if the NAS power supply or motherboard fails and all you have left that is functional are drives. If information is contained on a single drive, do you know the file system and how you can read it? If information is across multiple drives (ie. RAID 0, 5, 6) can you read that volume somehow - as a note, many NAS manufacturers will often change the implementation as their NAS models evolve, basically rendering a RAID volume inaccessible. Just some thought for a backup strategy where most people just think about the drive failures...

 

[Jockos]

I know this is digressing from the original topic but for your own backup strategy you also need to ask questions beyond "how do I recover if a drive fails?". As a backup strategy, using RAID volumes, you also need to ask what happens if the NAS power supply or motherboard fails and all you have left that is functional are drives. If information is contained on a single drive, do you know the file system and how you can read it? If information is across multiple drives (ie. RAID 0, 5, 6) can you read that volume somehow - as a note, many NAS manufacturers will often change the implementation as their NAS models evolve, basically rendering a RAID volume inaccessible. Just some thought for a backup strategy where most people just think about the drive failures...

Hopefully you store the data on (at least) two different machines (at different locations). If you NAS X goes down but discs survive, you put them in another NAS, format and then copy the files from NAS Y.

 

[sevo]

Enterprise drives are more reliable than consumer drives and work well as a backup drive.

Well, if you manage to find "enterprise" drives rated for office desktop or mobile office use, yes. But usually "enterprise" means server. And no maker I know of recommends server disks for general (home or office) use - on the contrary, quite a few warn against it.

Enterprise drives are designed for unattended stationary 24/7 operations in stable low temperature with no mechanical stress/shocks, and they reach their performance benchmarks under high parallel loads of multi-user databases and file servers. Some past enterprise disk types have even made it to notoriety for almost total failure in typical desktop use scenarios, failing after few weeks of power cycles or a single bump against the PC case.

Depending on how close your usage pattern gets to server room conditions they may be fine, a waste of money or even create additional risks. But the benefit will be small at the very best unless you operate a permanently powered heavy tower or rackmount PC in an air conditioned room.

I would absolutely avoid using them for backups - there is no benefit to them in such a very occasional use (and mobile external case) application, and plenty of added risks as server grade disks have reduced shock protection and are rated for a low number of on-off cycles.

 

[Photo_Smith]

OMG ... these pc issues will get me grey hair.


Q 1: Does anyone have an advice how to locate the data filling up the drive?
.

Try Tree size a freeware program that gives you a visual diagram of where the space is being used and the size of files/directories.

http://www.jam-software.com/freeware/

I like the professional version which has a 30 day trial
Good luck.....

 

[craygc]

Hopefully you store the data on (at least) two different machines (at different locations). If you NAS X goes down but discs survive, you put them in another NAS, format and then copy the files from NAS Y.

This (if I read you correctly, although that word 'format' worries me) is assuming that you have two identical NAS boxes, which is usually very unlikely for personal use - or even SOHO. Noting that a single RAID 'anything' volume definitely doesn't constitute "backup", the dialogue above was merely to highlight pitfalls that come with using RAID volumes in SOHO NAS boxes that people usually don't think about.

Personally, I run 2x RAID 1 in a single QNAP NAS and backup across the volumes (I also backup to another external drive as well :bang🙂 - and I know that none of the current QNAP models support a multi-drive RAID volume created on it. I know the file system is EXT3 and that I can read each single drive directly from the PC, or Mac in my case.

 

[icebear]

Try Tree size a freeware program that gives you a visual diagram of where the space is being used and the size of files/directories.

http://www.jam-software.com/freeware/

I like the professional version which has a 30 day trial
Good luck.....

Thanks for the link. It worked, I found 381 GB of 'RRbackups' but these folders are still not showing up when I select 'show hidden folders' under the view options in explorer. At least one step closer ...

 

[icebear]

Thanks for all the replies.
It looks like an enterprise 'always on' HD might not be a good choice for a consumer environment .

Do you have any recommendation
(3TB vs 4TB, number of discs per HD, 5600 rpm vs 7200, specific model to avoid?)
beyond getting a RAID setup, as all HD will eventually fail :bang: ?

As I am just doing LR adjustments similar do doging and burning and no fancy layer stuff, no video,
no gaming - speed is less important than reliability.

As for the importance, it's private use only, it's not that my family income depends on these data.

 

[Jockos]

This (if I read you correctly, although that word 'format' worries me) is assuming that you have two identical NAS boxes, which is usually very unlikely for personal use - or even SOHO. Noting that a single RAID 'anything' volume definitely doesn't constitute "backup", the dialogue above was merely to highlight pitfalls that come with using RAID volumes in SOHO NAS boxes that people usually don't think about.

Then you misunderstood my reply, you can have any model of boxes, in any RAID configuration.

Say Box 1 is a standalone NAS with two discs raided.
Box 2 is a computer with one disc.
Box 1 goes down but the discs survive.
Take Box 1 discs, format them, stick them in a new box, Box 1b.
Copy data from Box 2 to Box 1b.

Then you will have no issues regardless how the Box 1 discs were configured for raid or filesystems before the failure, as you suggested in your post.

 

[craygc]

Personally, I would only use Seagate or Western Digital drives, although I'm 100% WD these days. WD Red are rated well for long continuous use and have low power (and therefore heat) consumption. WD RE4 are probably even better in terms of life and performance, although more expensive. Both these drives are fine in consumer environments.

RAID (outside of RAID 0) addresses hardware failure ONLY, which is not a backup strategy - that is, what ever happens on one drive is immediately replicated on all other drives, be that good or bad. Backup is about taking temporal snap shots of you data - ie regular copying to another drive or to alternate drives and preferably, as Jockos mentioned, with off site copies as well.

 

[craygc]

Then you misunderstood my reply, you can have any model of boxes, in any RAID configuration.

Say Box 1 is a standalone NAS with two discs raided.
Box 2 is a computer with one disc.
Box 1 goes down but the discs survive.
Take Box 1 discs, format them, stick them in a new box, Box 1b.
Copy data from Box 2 to Box 1b.

Then you will have no issues regardless how the Box 1 discs were configured for raid or filesystems before the failure, as you suggested in your post.

I agree that would you are suggesting works but the primary issue is that in reality people tend to view a RAID config in a NAS as backup and survivability in the context of a drive failure (irrespective of having another drive copy of the data). Recovery from a drive failure is pretty straight forward; recovery from NAS hardware failure - in itself - can be a potential nightmare :bang:

 

[willie_901]

Fantom makes high-quality external drives. I use two, 2 TB units. One for automated hourly, weekly and monthly backups of my desktop computer. A second for all my raw files and original negative/transparency scans. The second unit is backed up regularly to a third 2 TB LaCie unit (from before I switched to Fantom).

 

[craygc]

Fantom makes high-quality external drives. I use two, 2 TB units. One for automated hourly, weekly and monthly backups of my desktop computer. A second for all my raw files and original negative/transparency scans. The second unit is backed up regularly to a third 2 TB LaCie unit (from before I switched to Fantom).

Neither of these companies make drives, they merely bundle up other drives in their cases. I know nothing about Fantom but would assume they're approach is similar to LaCie. LaCie uses any manufacturer's drives - just depends which ones are the cheapest this month. Furthermore, they use non-standard bridge boards, which means in the event of a enclosure failure you can't read the drive. Given this strategy of LaCie enclosures you're basically buying into trouble.

If you want an external drive, buy a case and a drive separately. At least that way you know what your getting and they're simple to assemble.

 

[bwcolor]

Enterprise drives are designed for unattended stationary 24/7 operations in stable low temperature with no mechanical stress/shocks, and they reach their performance benchmarks under high parallel loads of multi-user databases and file servers. Some past enterprise disk types have even made it to notoriety for almost total failure in typical desktop use scenarios, failing after few weeks of power cycles or a single bump against the PC case.


I would absolutely avoid using them for backups - there is no benefit to them in such a very occasional use (and mobile external case) application, and plenty of added risks as server grade disks have reduced shock protection and are rated for a low number of on-off cycles.

I've used the Western Digital Raid Edition drives for many years starting with their 500GB drives. I recognized that they are designed for constant use and I found the 5 year warranty to be reassuring. The higher failure rate with a low number of on-off cycles is news to me. I'm assuming that when you mention on-off, this is with respect to the head moving rather than the unit physically being powered down. Thanks for the information.

 

[craygc]

I've used the Western Digital Raid Edition drives for many years starting with their 500GB drives. Two of my three raid boxes are run 24/7, but I did not know that the uneven usage would increase the chances of failure.

Drives build specifically for RAID deployments are really optimised around fast error recovery and configurable error recovery to avoid drives dropping off a volume due to long recovery cycles; low power, or at least intelligent power, usage; better balanced for less vibration and noise; and non-aggressive head parking, these are the main differences. They work well for 24/7 usage and performance and life is pretty much independent from uneven usage and load over their lifetime.