Sunday, August 11, 2013

Large Capacity Drives and Windows: "746GB of Doom"

You probably ended up here from a Google Search.  Good.

It's extremely probable that you bought a snazzy new 3TB or greater drive, in the hopes of loading all of your "legally purchased" movies and music to it, so you can stream to all of your mobile devices in a nirvana of media gold.

Only problem is, Windows swears your drive only has 746-ish GB of storage.  So you assume the drive is defective.  It's not.  In fact, it's your hardware that's the problem, and we really should blame computer and accessory manufacturers (such as Seagate, Western Digital, AMD, Intel, etc) for not having what I consider necessary disclaimers.

I'm a big "teach a man how to fish" fan.  So I'm going to first explain why you're having a problem, then I'll give you the definitive solution.  I'm almost confident that you've browsed many forums, help articles, and tech sites that give you plenty of "fixes", only to find none of them work.  That's why I have to teach you why the issue is an issue before I give the solution because, there isn't a single solution for everyone.  It depends on your hardware.

In a nutshell: your hardware is too stupid to read the capacity properly because it's stuck in the Dark Ages.

"but I just bought this computer!!" good.  That'll make it easier to fix your issue, because you'll have less steps.  On the other step, if your computer was top notch back when Apple was on the verge of going out of business, I suggest you upgrade.  There's no reason not to.  Desktops can be had for under $300, laptops for under $600.  You don't need state-of-the-art, but you do need to be recent.  Here's why.

Storage History
Back in the 90's the concept of cylinders, sectors, and blocks was a very simple and easy method of "telling" a drive how much storage it had.  Most common users never needed to care about this metric, but if you needed to run tools, you might have heard the term "bad sector".  This indicated a problem with one of the parts of the drive that data was stored on, and just about every operating system had tools to correct these errors.  Meanwhile, the BIOS (think of it as the "brain" of your computer.  It tells your hardware and software how to talk to each other) did not advance rapidly to consider higher levels of storage.  It was not unheard of to think that 160MB of disk storage was a big deal, and it was....back when programs barely took 1MB.  Now, most programs can easily take up to 500MB to 1GB or more depending on the program.

Larger disks yielded opportunities that people didn't have before.  Unfortunately, the hardware and software did not advance fast enough to keep up with the need for greater storage.  Windows Millennium (for those that can remember) and operating systems at that time could barely tap more than a certain amount of storage, and were limited to what file sizes could be accessed.  Windows XP and Windows 2000 helped this along a bit, but it was really (ironically) Windows Vista that opened the door to more possibilities.  It was the first operating system where the idea of UEFI was discussed as the next evolution of BIOS.

Where standard BIOS was akin to rubbing sticks together to make fire - functional but primitive - UEFI is more akin to using urine as an electrical conductor; natural, though still crude, and requiring very little resource to manage.  It allows the hardware to address more storage than the older BIOS, and as such, allows computers to make use of the greater hardware.

Then you run into the motherboard itself.  Older motherboards supported older connector types; any computer purchased within the past decade should easily be able to tap into SATA connections.  SATA provides a much easier way of connecting drives to the board, and faster throughput speeds; but beyond the connector type, there are different SATA flavors.  A large capacity drive is likely going to support SATA III, which boards within the past 3 years should support.  That means if your computer is older than that, your drive will work, but not nearly as fast as it could.  Some drives may even compel you to apply jumper settings to essentially "throttle" the drive own to the lower speed your board supports.  Rather than fumbling with jumpers, let me give you a small tidbit of advice: just upgrade.  Seriously.

While UEFI deals with and relieves the hardware challenges, you still have the software challenges.  The reason for this: The Master Boot Record, or MBR.  Known to old hardware technicians for decades as the easiest, simplest and least brain taxing method of configuring drive partitioning, it also was limiting to the operating system, as the MBR did not allow for supporting disk volume sizes greater than 1.5TB.  What throws people is the name: Master "Boot" Record, which still limits you even if you don't plan to boot from the disk.  Thus the GUID Partition Table, or GPT, was introduced.  I would point out that GPT was supported in Apple's Mac OS for quite some time, but rarely used, since UEFI did not get introduced but within the past few years.  Windows did not catch up with both until Vista, but Windows by design wants to do everything with MBR because the operating system leans towards that by default.  Even in Windows 8 it is not smart enough to detect drive size when making the decision.

The net effect of the above is a crapshoot that may reveal itself as a 746-ish GB drive.  Even though it is supposed to support 3TB/4TB.

The Solution
The answer to this depends on your hardware.  I will first tell you what WON'T work, then talk about what will.  All of this assumes you have a newer computer.  If you have an older computer, half of these suggestions won't work for you.  You need to upgrade.  If you're running an operating system lower than Windows Vista Service Pack 2, you need to upgrade.  (and frankly, there's no excuse for you not being on Windows 7 by now.)

First, don't bother trying with an external housing or USB-SATA cable.  Most people will be tempted to do this to try and set the drive up, or perhaps they plan to leave it there, or perhaps they don't want to put it in a computer.  Don't do this.  Why?  While you could find an enclosure that would support the drive, they're significantly more expensive, for less value.  Also, you want to make sure you're truly seeing the drive for what it's supposed to be, and the best way to do that is to install it directly into the computer.  This isn't to say it's impossible to work external, but the #2 edict of troubleshooting is process of elimination: remove all other variables, and whatever you're left with must be the cause.  (FYI, the #1 edict is Occam's Razor: The easiest answer is usually the correct one.)

This doesn't apply to a NAS device, because a NAS has a built-in operating system and hardware and, provided the box marks it so, should properly support the larger sizes.

Second, DON'T format the drive if it's not installed in the computer.  You can correct this but it's just wasting time.  Again, you want to be directly connected to make sure the right size is reported.

After considering the above, you should do the below, in the exact order.

First, install all Windows and hardware updates.  Even if you have to go to the manufacturer directly (which I had to do.  More on that later).  Why?  Because often, the hardware vendor will have shipped a device without UEFI and with an older chipset that doesn't support large storage, but publish an update so you can add the support.  You'll want to make sure all of these updates are applied, especially BIOS updates.

For AMD Chipsets you will almost certainly need to download the latest drivers from their site.  DO NOT just trust the computer manufacturer's website.  In my experience, which was a Gateway, they did not include chipset drivers in the Catalyst package, even though it said it was included.  The chipset drivers were not added until I got the package from AMD directly.  You need to make sure that your driver reads as AHCI.  This is critical.  If you aren't able to locate an AHCI driver, your board may not support the larger drives.

For Intel Chipsets you will likely need to make sure that Intel's Rapid Storage Technology software is installed.  This software essentially acts as a driver for the chipset that allows it to address larger storage pieces.  Many online forums have alluded to the need for this.  But make sure you actually have Intel (it should be a sticker on the computer), because if it's not Intel and you use Intel's driver scan, you will blue screen.

Second, install the drive into the computer. If it's a small form factor machine you may need to permanently (or temporarily) remove the CD/DVD drive.  In my case I have no need for an optical drive in the computer since external ones are around $20 and will do for the one-off disk access I need, so I removed the optical drive and got special mounts to load the storage drive in its place.  That gives me a primary disk of 500GB and a storage disk of 3TB (2.8TB available).  MAKE SURE YOUR COMPUTER IS POWERED OFF AND UNPLUGGED FOR A FEW MINUTES BEFORE DOING THIS!!

Third, boot into the BIOS.  Different manufacturers use different keys to do this.  It might be Delete, might be F2, might be F6.  Should tell you near the bottom when it's booting up.  The BIOS should have a blue-ish look; this is how you know it's UEFI.  If it's jet black with ugly yellow or gray or green text, you're on the older BIOS and you need to upgrade.  A BIOS upgrade is always downloaded from the computer manufacturer, and only takes a few minutes to do.  Also, if you have UEFI, make sure AHCI is enabled on your disks.  It should have such a setting though it may have a funny name.  Again, consult your computer manufacturer for questions about this.

Fourth, reboot the computer and let it load into Windows.  At this point you should be ready to set your drive up if everything was done correctly.  You'll need to go into Computer Management/Disk Management; Windows 8 has a different way of going about getting there, but if you can right-click on Computer and hit Manage, that step should work on every Windows OS.  You may get prompted to continue, go ahead and accept.  Then select Disk Management under Storage, and let it load.  You should then see something similar (but not exactly identical) to the below:
Notice that while my Disk is the proper size, I have three separate "chunks".  These are partitions that I've created to store different data.  The data lives on the same physical storage, but they are accessed individually.  In your case, you should end up with a Disk of the proper size, with a single "Unallocated" chunk.  As long as the size is close to correct, you're good to go. It will always be slightly lower than the storage on the sticker due to the way storage is calculated; do some searching for more info on that because it's TLDR material.

If you've gotten this far, congrats.  You've resolved your issue.  At this point I'm assuming you know how to do partitioning and drive setup, but if you don't, there are plenty of resources on the web about it.  This is only to resolve the short space issue with larger disks.