{"id":115,"date":"2014-07-25T16:51:41","date_gmt":"2014-07-25T14:51:41","guid":{"rendered":"http:\/\/46.105.150.90\/tstnunix\/?p=115"},"modified":"2014-07-25T16:51:41","modified_gmt":"2014-07-25T14:51:41","slug":"too-hard-to-be-above-2tb","status":"publish","type":"post","link":"https:\/\/nunix.fr\/?p=115","title":{"rendered":"Too hard to drive Advanced Format"},"content":{"rendered":"

I’ve been struggling for quiet a long time about a problem with a Western Digital 3TB hard drive. I let it aside as, back in time, I found a quick workaround which is not efficient on the long term: using windows to partition the disk, and use windows to read\/write on it.<\/p>\n

I’ve finally decided to tackle this inconsistency and looked desperately for a satisfying solution. As I’ve been looking for the solution for more than few dozen minutes, this page might help other people \ud83d\ude42<\/p>\n

Problem is: When plugged through USB enclosure, the kernel sees 1565565872 sectors of 512 bytes, where in fact there are 1565565872 sectors of 4096 bytes in my hard drive. I’m still a bit confused with the responsabilities. Apparently, Western Digital firmware doesn’t respond well to some commands, reporting wrong values. But the kernel is not able to circumvent these wrong values despite accessible workaround to retrieve the valid values.<\/p>\n

In the following, we compare what you can encounter when plugging-in your hard drive through USB enclosure, or through SATA enclosure. I take advantage of the following to also present few debugging tips for SCSI or ATA commands. <\/p>\n

USB Enclosure<\/h2>\n

Through dmesg, you observe the following logs when plugging-in the device (here plugged as a USB device). Kernel reports a 801GB hard-drive, which is incorrect.
{codecitation class=\u00a0\u00bbbrush: shell;highlight: [11];\u00a0\u00bb}<\/p>\n

[ 3191.115145] usb 3-2: new high-speed USB device number 3 using xhci_hcd
[ 3191.273809] usb 3-2: New USB device found, idVendor=04fc, idProduct=0c25
[ 3191.273821] usb 3-2: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[ 3191.273826] usb 3-2: Product: USB to Serial-ATA bridge
[ 3191.273831] usb 3-2: Manufacturer: Sunplus Technology Inc.
[ 3191.273835] usb 3-2: SerialNumber: WDC WD30EZ\u00a0\u00a0\u00a0\u00a0 WD-WCC070365571
[ 3191.276667] usb-storage 3-2:1.0: USB Mass Storage device detected
[ 3191.276876] scsi8 : usb-storage 3-2:1.0
[ 3192.212163] scsi 8:0:0:0: Direct-Access\u00a0\u00a0\u00a0\u00a0 WDC WD30 EZRX-00AZ6B0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 PQ: 0 ANSI: 2
[ 3192.214531] sd 8:0:0:0: Attached scsi generic sg2 type 0
[ 3192.215147] sd 8:0:0:0: [sdc] 1565565872 512-byte logical blocks: (801 GB\/746 GiB)
[ 3192.217152] sd 8:0:0:0: [sdc] Write Protect is off
[ 3192.217164] sd 8:0:0:0: [sdc] Mode Sense: 38 00 00 00
[ 3192.221024] sd 8:0:0:0: [sdc] No Caching mode page found
[ 3192.221035] sd 8:0:0:0: [sdc] Assuming drive cache: write through
[ 3192.273232]\u00a0 sdc: sdc1
[ 3192.279097] sd 8:0:0:0: [sdc] Attached SCSI disk<\/p>\n

{\/codecitation}<\/p>\n

The disk has a GPT partition table, not really supported by fdisk.
{codecitation class=\u00a0\u00bbbrush: shell;\u00a0\u00bb}<\/p>\n

$ fdisk -l \/dev\/sdb<\/p>\n

Disk \/dev\/sdb: 746.5 GiB, 801569726464 bytes, 1565565872 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical\/physical): 512 bytes \/ 512 bytes
I\/O size (minimum\/optimal): 512 bytes \/ 512 bytes
Disklabel type: gpt
Disk identifier: 2FD103AE-2893-4BC1-A4A0-A9D747B0013E<\/p>\n

Device\u00a0\u00a0\u00a0 Start\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 End\u00a0\u00a0 Size Type
\/dev\/sdb1\u00a0\u00a0\u00a0 34\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 262177\u00a0\u00a0 128M Microsoft reserved
{\/codecitation}<\/p>\n

Using gdisk instead gives us better insight. The utility thinks that the last usable sector is 1565565838, whereas the partition’s end is at sector 5860532223 (the partition has in addition some small issues that have been corrected. It’s independent from the problem with advanced format).<\/p>\n

{codecitation class=\u00a0\u00bbbrush: shell;highlight: [17];\u00a0\u00bb}
$ gdisk -l \/dev\/sdb
GPT fdisk (gdisk) version 0.8.10<\/p>\n

The protective MBR’s 0xEE partition is oversized! Auto-repairing.<\/p>\n

Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present<\/p>\n

Found valid GPT with protective MBR; using GPT.<\/p>\n

Warning! Secondary partition table overlaps the last partition by
4294966385 blocks!
You will need to delete this partition or resize it in another utility.
Disk \/dev\/sdb: 1565565872 sectors, 746.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 2FD103AE-2893-4BC1-A4A0-A9D747B0013E
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1565565838
Partitions will be aligned on 8-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)<\/p>\n

Number\u00a0 Start (sector)\u00a0\u00a0\u00a0 End (sector)\u00a0 Size\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Code\u00a0 Name
1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 34\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 262177\u00a0\u00a0 128.0 MiB\u00a0\u00a0 0C01\u00a0 Microsoft reserved …
2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 264192\u00a0\u00a0\u00a0\u00a0\u00a0 5860532223\u00a0\u00a0 2.7 TiB\u00a0\u00a0\u00a0\u00a0 0700\u00a0 Basic data partition
{\/codecitation}<\/p>\n

If you run hdparm, you also get confusing info: the utility reports the correct values: 512 bytes for the logical sector size, but 4096 for the physical one.
{codecitation class=\u00a0\u00bbbrush: shell;highlight: [23,24];\u00a0\u00bb}
$ hdparm -I \/dev\/sdb<\/p>\n

\/dev\/sdb:
SG_IO: bad\/missing sense data, sb[]:\u00a0 70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00<\/p>\n

ATA device, with non-removable media
Model Number:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 WDC WD30EZRX-00AZ6B0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0
Serial Number:\u00a0\u00a0\u00a0\u00a0\u00a0 \u0001\ufffd@C70365571
Firmware Revision:\u00a0 80.00A80
Transport:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
Supported: 8 7 6 5
Likely used: 8
Configuration:
Logical\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0max\u00a0\u00a0 \u00a0current
cylinders\u00a0\u00a0 \u00a017218\u00a0\u00a0 \u00a016383
heads\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a00\u00a0\u00a0 \u00a016
sectors\/track\u00a0\u00a0 \u00a0128\u00a0\u00a0 \u00a063
—
CHS current addressable sectors:\u00a0\u00a0 16514064
LBA\u00a0\u00a0\u00a0 user addressable sectors:\u00a0 268435455
LBA48\u00a0 user addressable sectors: 5860533168
Logical\u00a0 Sector size:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 512 bytes
Physical Sector size:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 4096 bytes
Logical Sector-0 offset:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0 bytes
device size with M = 1024*1024:\u00a0\u00a0\u00a0\u00a0 2861588 MBytes
device size with M = 1000*1000:\u00a0\u00a0\u00a0\u00a0 3000592 MBytes (3000 GB)
cache\/buffer size\u00a0 = unknown
Nominal Media Rotation Rate: 5400
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec’d by Standard, with device specific minimum
R\/W multiple sector transfer: Max = 16\u00a0\u00a0 \u00a0Current = 16
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns\u00a0 IORDY flow control=120ns
Commands\/features:
Enabled\u00a0\u00a0 \u00a0Supported:
*\u00a0\u00a0 \u00a0SMART feature set
Security Mode feature set
*\u00a0\u00a0 \u00a0Power Management feature set
*\u00a0\u00a0 \u00a0Write cache
*\u00a0\u00a0 \u00a0Look-ahead
*\u00a0\u00a0 \u00a0Host Protected Area feature set
*\u00a0\u00a0 \u00a0WRITE_BUFFER command
*\u00a0\u00a0 \u00a0READ_BUFFER command
*\u00a0\u00a0 \u00a0NOP cmd
*\u00a0\u00a0 \u00a0DOWNLOAD_MICROCODE
Power-Up In Standby feature set
*\u00a0\u00a0 \u00a0SET_FEATURES required to spinup after power up
SET_MAX security extension
*\u00a0\u00a0 \u00a048-bit Address feature set
*\u00a0\u00a0 \u00a0Mandatory FLUSH_CACHE
*\u00a0\u00a0 \u00a0FLUSH_CACHE_EXT
*\u00a0\u00a0 \u00a0SMART error logging
*\u00a0\u00a0 \u00a0SMART self-test
*\u00a0\u00a0 \u00a0General Purpose Logging feature set
*\u00a0\u00a0 \u00a064-bit World wide name
*\u00a0\u00a0 \u00a0WRITE_UNCORRECTABLE_EXT command
*\u00a0\u00a0 \u00a0{READ,WRITE}_DMA_EXT_GPL commands
*\u00a0\u00a0 \u00a0Segmented DOWNLOAD_MICROCODE
*\u00a0\u00a0 \u00a0Gen1 signaling speed (1.5Gb\/s)
*\u00a0\u00a0 \u00a0Gen2 signaling speed (3.0Gb\/s)
*\u00a0\u00a0 \u00a0Gen3 signaling speed (6.0Gb\/s)
*\u00a0\u00a0 \u00a0Native Command Queueing (NCQ)
*\u00a0\u00a0 \u00a0Host-initiated interface power management
*\u00a0\u00a0 \u00a0Phy event counters
*\u00a0\u00a0 \u00a0NCQ priority information
DMA Setup Auto-Activate optimization
*\u00a0\u00a0 \u00a0Software settings preservation
*\u00a0\u00a0 \u00a0SMART Command Transport (SCT) feature set
*\u00a0\u00a0 \u00a0SCT Write Same (AC2)
*\u00a0\u00a0 \u00a0SCT Features Control (AC4)
*\u00a0\u00a0 \u00a0SCT Data Tables (AC5)
unknown 206[12] (vendor specific)
unknown 206[13] (vendor specific)
Security:
Master password revision code = 65534
supported
not\u00a0\u00a0 \u00a0enabled
not\u00a0\u00a0 \u00a0locked
not\u00a0\u00a0 \u00a0frozen
not\u00a0\u00a0 \u00a0expired: security count
supported: enhanced erase
more than 508min for SECURITY ERASE UNIT. more than 508min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 50014ee208471388
NAA\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0: 5
IEEE OUI\u00a0\u00a0 \u00a0: 0014ee
Unique ID\u00a0\u00a0 \u00a0: 208471388
Checksum: incorrect (0x97), expected 0x69
{\/codecitation}<\/p>\n

A final verification of the values seen by the kernel:<\/p>\n

{codecitation class=\u00a0\u00bbbrush: shell;\u00a0\u00bb}<\/p>\n

$ cat \/sys\/class\/block\/sdb\/size
1565565872
$ cat \/sys\/class\/block\/sdb\/queue\/logical_block_size
512
$ cat \/sys\/class\/block\/sdb\/queue\/physical_block_size
512<\/p>\n

{\/codecitation}<\/p>\n

SATA enclosure<\/h2>\n

In the following, the hard drive is plugging-in through SATA wires. dmesg reports a disk with 4096-byte physical blocks, which is correct. The kernel will then address the blocks through one of the 5860531055 512-byte logical blocks.<\/p>\n

{codecitation class=\u00a0\u00bbbrush: shell;highlight: [6];\u00a0\u00bb}
[\u00a0\u00a0\u00a0 1.643704] ata2.00: HPA detected: current 5860531055, native 5860533168
[\u00a0\u00a0\u00a0 1.643709] ata2.00: ATA-8: WDC WD30EZRX-00AZ6B0, 80.00A80, max UDMA\/133
[\u00a0\u00a0\u00a0 1.643712] ata2.00: 5860531055 sectors, multi 16: LBA48 NCQ (depth 0\/32)
[\u00a0\u00a0\u00a0 1.665744] ata2.00: configured for UDMA\/133
[\u00a0\u00a0\u00a0 1.665859] scsi 1:0:0:0: Direct-Access\u00a0\u00a0\u00a0\u00a0 ATA\u00a0\u00a0\u00a0\u00a0\u00a0 WDC WD30EZRX-00A 80.0 PQ: 0 ANSI: 5
[\u00a0\u00a0\u00a0 1.665979] sd 1:0:0:0: [sda] 5860531055 512-byte logical blocks: (3.00 TB\/2.72 TiB)
[\u00a0\u00a0\u00a0 1.665982] sd 1:0:0:0: [sda] 4096-byte physical blocks
[\u00a0\u00a0\u00a0 1.666017] sd 1:0:0:0: Attached scsi generic sg0 type 0
[\u00a0\u00a0\u00a0 1.666046] sd 1:0:0:0: [sda] Write Protect is off
[\u00a0\u00a0\u00a0 1.666049] sd 1:0:0:0: [sda] Mode Sense: 00 3a 00 00
[\u00a0\u00a0\u00a0 1.666076] sd 1:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn’t support
DPO or FUA
[\u00a0\u00a0\u00a0 1.670405] scsi 3:0:0:0: CD-ROM\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 HL-DT-ST DVDRAM GSA-H66N\u00a0 CB00 PQ: 0 ANSI: 5
[\u00a0\u00a0\u00a0 1.708665] Alternate GPT is invalid, using primary GPT.
[\u00a0\u00a0\u00a0 1.708677]\u00a0 sda: sda1
[\u00a0\u00a0\u00a0 1.708962] sd 1:0:0:0: [sda] Attached SCSI disk
{\/codecitation}<\/p>\n

Fdisk utility still has problem reading GPT partition table, but the disk size is correct.<\/p>\n

{codecitation class=\u00a0\u00bbbrush: shell;highlight: [];\u00a0\u00bb}
$ fdisk -l \/dev\/sda
The backup GPT table is corrupt, but the primary appears OK, so that will be used.<\/p>\n

Disk \/dev\/sda: 2.7 TiB, 3000591900160 bytes, 5860531055 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical\/physical): 512 bytes \/ 4096 bytes
I\/O size (minimum\/optimal): 4096 bytes \/ 4096 bytes
Disklabel type: gpt
Disk identifier: 2FD103AE-2893-4BC1-A4A0-A9D747B0013E
Partition 1 does not start on physical sector boundary.<\/p>\n

Device\u00a0\u00a0\u00a0 Start\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 End\u00a0\u00a0 Size Type
\/dev\/sda1\u00a0\u00a0\u00a0 34\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 262177\u00a0\u00a0 128M Microsoft reserved
{\/codecitation}<\/p>\n

The gdisk utility is able to read the partition, despite small errors that are independent from the physical and logical block size.<\/p>\n

{codecitation class=\u00a0\u00bbbrush: shell;highlight: [21];\u00a0\u00bb}
$ gdisk -l \/dev\/sda
GPT fdisk (gdisk) version 0.8.8<\/p>\n

Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.<\/p>\n

Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: damaged<\/p>\n

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************<\/p>\n

Warning! Secondary partition table overlaps the last partition by
4294966385 blocks!
You will need to delete this partition or resize it in another utility.
Disk \/dev\/sda: 5860531055 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 2FD103AE-2893-4BC1-A4A0-A9D747B0013E
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1565565838
Partitions will be aligned on 8-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)<\/p>\n

Number\u00a0 Start (sector)\u00a0\u00a0\u00a0 End (sector)\u00a0 Size\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Code\u00a0 Name
1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 34\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 262177\u00a0\u00a0 128.0 MiB\u00a0\u00a0 0C01\u00a0 Microsoft reserved part
2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 264192\u00a0\u00a0\u00a0\u00a0\u00a0 5860532223\u00a0\u00a0 2.7 TiB\u00a0\u00a0\u00a0\u00a0 0700\u00a0 Basic data partition
{\/codecitation}<\/p>\n

And kernel sees correct values. The number of sectors has been adapted by the kernel to now report 5860531055 sectors.
{codecitation class=\u00a0\u00bbbrush: shell;highlight: [11];\u00a0\u00bb}
$ cat \/sys\/class\/block\/sda\/size
5860531055
$ cat \/sys\/class\/block\/sda\/queue\/logical_block_size
512
$ cat \/sys\/class\/block\/sda\/queue\/physical_block_size
4096
{\/codecitation}<\/p>\n

Digging into the problem<\/h2>\n

I’m still a bit confused in the root cause of the issue. WD digital seems to report improper values although some utility tools (such as hdparm) are able to retrieve the correct size of the logical and physical sectors. In addition, Windows correctly mounts the device and size is correct with this OS. I’m trying in the following to narrow down the problem. The difference seen by the kernel between USB enclosure and SATA enclosure can be summarize as the following log lines:<\/p>\n

USB enclosure:\u00a0 scsi 8:0:0:0: Direct-Access\u00a0\u00a0\u00a0\u00a0 WDC WD30 EZRX-00AZ6B0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 PQ: 0 ANSI: 2
SATA enclosure: scsi 1:0:0:0: Direct-Access\u00a0\u00a0\u00a0\u00a0 ATA\u00a0\u00a0\u00a0\u00a0\u00a0 WDC WD30EZRX-00A 80.0 PQ: 0 ANSI: 5<\/p>\n

There are several ways to collect the disk size at a low level. Through ATA Identify command, or through an SCSI read capacity command.<\/p>\n

ATA Identify command<\/h3>\n

One way to obtain the values of the idenfity command is to use smartctl. In the following, the 106th word is 0x6003. According to specs (7.16.7.54 from T13\/1699-D AT Attachment 8 – ATA\/ATAPI Command Set (ATA8-ACS)<\/a>), it means that \u00ab\u00a0the device has more than one logical sector per physical sector and bits (3:0) of word 106 are valid\u00a0\u00bb. In other words, it means that the physical block size is height times the logical block size, hence 4096.<\/p>\n

{codecitation class=\u00a0\u00bbbrush:shell;highlight:[46];\u00a0\u00bb}<\/p>\n

$ smartctl -a -r ioctl,2 \/dev\/sdc
smartctl 6.2 2014-07-16 r3952 [x86_64-linux-3.15.5-200.fc20.x86_64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org<\/p>\n

USB ID = 0x04fc:0x0c25 (0x103)<\/p>\n

REPORT-IOCTL: Device=\/dev\/sdc Command=IDENTIFY DEVICE
Input:\u00a0\u00a0 FR=…., SC=0x01, LL=…., LM=…., LH=…., DEV=…., CMD=0xec IN
[: f8 00 22 10 01 00 01 00 00 00 a0 ec ]
scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0\u00a0 duration=6 milliseconds\u00a0 resid=0
Incoming data, len=512 [only first 256 bytes shown]:
00\u00a0\u00a0\u00a0\u00a0 7a 42 ff 3f 37 c8 10 00\u00a0 00 00 00 00 3f 00 00 00\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
10\u00a0\u00a0\u00a0\u00a0 00 00 00 00 20 20 20 20\u00a0 57 20 2d 44 43 57 30 43\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
20\u00a0\u00a0\u00a0\u00a0 30 37 36 33 35 35 31 37\u00a0 00 00 00 00 32 00 30 38\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
30\u00a0\u00a0\u00a0\u00a0 30 2e 41 30 30 38 44 57\u00a0 20 43 44 57 30 33 5a 45\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
40\u00a0\u00a0\u00a0\u00a0 58 52 30 2d 41 30 36 5a\u00a0 30 42 20 20 20 20 20 20\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
50\u00a0\u00a0\u00a0\u00a0 20 20 20 20 20 20 20 20\u00a0 20 20 20 20 20 20 10 80\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
60\u00a0\u00a0\u00a0\u00a0 00 00 00 2f 01 40 00 00\u00a0 00 00 07 00 ff 3f 10 00\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
70\u00a0\u00a0\u00a0\u00a0 3f 00 10 fc fb 00 10 01\u00a0 ff ff ff 0f 00 00 07 00\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
80\u00a0\u00a0\u00a0\u00a0 03 00 78 00 78 00 78 00\u00a0 78 00 00 00 00 00 00 00\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
90\u00a0\u00a0\u00a0\u00a0 00 00 00 00 00 00 1f 00\u00a0 0e 17 02 00 44 00 40 00\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
a0\u00a0\u00a0\u00a0\u00a0 fe 01 00 00 6b 74 61 75\u00a0 23 41 69 74 41 b4 23 41\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
b0\u00a0\u00a0\u00a0\u00a0 7f 40 ff 00 ff 00 00 00\u00a0 fe ff 00 00 00 00 00 00\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
c0\u00a0\u00a0\u00a0\u00a0 00 00 00 00 00 00 00 00\u00a0 6f 9b 50 5d 01 00 00 00\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
d0\u00a0\u00a0\u00a0\u00a0 00 00 00 00 03 60 00 00\u00a0 01 50 e2 4e 47 08 88 13\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
e0\u00a0\u00a0\u00a0\u00a0 00 00 00 00 00 00 00 00\u00a0 00 00 00 00 00 00 1c 40\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
f0\u00a0\u00a0\u00a0\u00a0 1c 40 00 00 00 00 00 00\u00a0 00 00 00 00 00 00 00 00\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
[Duration: 0.006s]
REPORT-IOCTL: Device=\/dev\/sdc Command=IDENTIFY DEVICE returned 0<\/p>\n

===== [IDENTIFY DEVICE] DATA START (BASE-16) =====
000-015: 7a 42 ff 3f 37 c8 10 00 00 00 00 00 3f 00 00 00 |zB.?7…….?…|
016-031: 00 00 00 00 20 20 20 20 57 20 2d 44 43 57 30 43 |….\u00a0\u00a0\u00a0 W -DCW0C|
032-047: 30 37 36 33 35 35 31 37 00 00 00 00 32 00 30 38 |07635517….2.08|
048-063: 30 2e 41 30 30 38 44 57 20 43 44 57 30 33 5a 45 |0.A008DW CDW03ZE|
064-079: 58 52 30 2d 41 30 36 5a 30 42 20 20 20 20 20 20 |XR0-A06Z0B\u00a0\u00a0\u00a0\u00a0\u00a0 |
080-095: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 10 80 |\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ..|
096-111: 00 00 00 2f 01 40 00 00 00 00 07 00 ff 3f 10 00 |…\/.@…….?..|
112-127: 3f 00 10 fc fb 00 10 01 ff ff ff 0f 00 00 07 00 |?……………|
128-143: 03 00 78 00 78 00 78 00 78 00 00 00 00 00 00 00 |..x.x.x.x…….|
144-159: 00 00 00 00 00 00 1f 00 0e 17 02 00 44 00 40 00 |…………D.@.|
160-175: fe 01 00 00 6b 74 61 75 23 41 69 74 41 b4 23 41 |….ktau#AitA.#A|
176-191: 7f 40 ff 00 ff 00 00 00 fe ff 00 00 00 00 00 00 |.@…………..|
192-207: 00 00 00 00 00 00 00 00 6f 9b 50 5d 01 00 00 00 |……..o.P]….|
208-223: 00 00 00 00 03 60 00 00 01 50 e2 4e 47 08 88 13 |…..`…P.NG…|
224-239: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1c 40 |……………@|
240-255: 1c 40 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |.@…………..|
256-271: 21 00 00 00 00 00 00 00 00 00 fe 16 44 01 00 00 |!………..D…|
272-287: 00 00 00 00 00 00 00 00 00 00 00 00 04 00 00 00 |…………….|
288-303: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|
304-319: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|
320-335: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|
336-351: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|
352-367: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|
368-383: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|
384-399: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|
400-415: 00 00 00 00 00 00 00 00 00 00 00 00 35 30 00 00 |…………50..|
416-431: 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00 00 |…@…………|
432-447: 00 00 18 15 00 00 00 00 00 00 00 00 3e 10 00 00 |…………>…|
448-463: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|
464-479: 00 00 00 00 01 00 00 10 00 00 00 00 00 00 00 00 |…………….|
480-495: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|
496-511: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 a5 35 |……………5|
===== [IDENTIFY DEVICE] DATA END (512 Bytes) =====<\/p>\n

=== START OF INFORMATION SECTION ===
Model Family:\u00a0\u00a0\u00a0\u00a0 Western Digital Caviar Green (AF, SATA 6Gb\/s)
Device Model:\u00a0\u00a0\u00a0\u00a0 WDC WD30EZRX-00AZ6B0
Serial Number:\u00a0\u00a0\u00a0 WD-WCC070365571
LU WWN Device Id: 5 0014ee 208471388
Firmware Version: 80.00A80
User Capacity:\u00a0\u00a0\u00a0 3,000,591,900,160 bytes [3.00 TB]
Sector Sizes:\u00a0\u00a0\u00a0\u00a0 512 bytes logical, 4096 bytes physical
Rotation Rate:\u00a0\u00a0\u00a0 5400 rpm
Device is:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 In smartctl database [for details use: -P show]
ATA Version is:\u00a0\u00a0 ATA8-ACS (minor revision not indicated)
SATA Version is:\u00a0 SATA 3.0, 6.0 Gb\/s (current: 1.5 Gb\/s)
Local Time is:\u00a0\u00a0\u00a0 Mon Jul 28 18:26:54 2014 CEST
SMART support is: Available – device has SMART capability.
SMART support is: Enabled
….
….<\/p>\n

{\/codecitation}<\/p>\n

SCSI Read capacity command<\/h3>\n

When the disk is plugged through the USB enclosure, the scsi command read capacity is used by the kernel driver. The problem is that the disk responds with inconsistent values:\u00a0 0x5d509b6e sectors of size 0x200, whereas there are supposed to be 0x15d509b6e sectors on the disk (entry [7] points to that interpretation). Regarding the specifications (check [8]), there are two READ CAPACITY commands. One called READ CAPACITY(10) and the other called READ CAPACITY(16). We observe that the former contains \u00ab\u00a0LOGICAL BLOCKS PER PHYSICAL BLOCK EXPONENT\u00a0\u00bb element in return. But this command is not implemented in the WD disk (apparently?). Therefore, the kernel has no possibility to infer that the physical block size is greater than the logical block size.<\/p>\n

{codecitation class=\u00a0\u00bbbrush: shell;highlight: [4,9];\u00a0\u00bb}
# sg_readcap -v \/dev\/sdc
read capacity (10) cdb: 25 00 00 00 00 00 00 00 00 00
Read Capacity results:
Last logical block address=1565563758 (0x5d509b6e), Number of blocks=1565563759
Logical block length=512 bytes
Hence:
Device size: 801568644608 bytes, 764435.4 MiB, 801.57 GB<\/p>\n

# sg_readcap –raw \/dev\/sdc | xxd
0000000: 5d50 9b6e 0000 0200\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ]P.n….<\/p>\n

# sg_readcap –long -v\u00a0 \/dev\/sdc
read capacity (16) cdb: 9e 10 00 00 00 00 00 00 00 00 00 00 00 20 00 00
read capacity (16):\u00a0 Fixed format, current;\u00a0 Sense key: Illegal Request
Additional sense: Invalid command operation code
READ CAPACITY (16) not supported
{\/codecitation}<\/p>\n

Tips used during debugging<\/h3>\n

We have used tools to execute raw commands at the disk level, such as sg_readcap that is part of the\u00a0 Linux sg3_utils package [5]. It might be good to know alternatives if you don’t have access to these tools. To capture raw USB, look at [6], and use graphical tools such as vusb-analyzer<\/a>. They do ease the investigation. I’ve been able to retrieve the identify ATA command and the read capacity scsi command.<\/p>\n

To retrieve the values directly (without using external programs), you can write your own code and use ioctl system calls such as in the following. If you prefer to use Python language, wrappers exists for the ioctl [9, 10].<\/p>\n

{codecitation class=\u00a0\u00bbbrush:c;\u00a0\u00bb}<\/p>\n

int fd;<\/p>\n


\/**
 * Wrapper for ioctl filling a long value
 **\/
unsigned long request2(char * name, int mode){
\tunsigned long nr;
\tif (0 == ioctl(fd, mode, &nr))
\t\tprintf(\"%s: size=%u\\n\", name, nr);
\t\t
\treturn nr;
}

int main (int argc, char **argv){
\tchar * name = \"\/dev\/sdc\";
\tstruct hd_geometry hdg;
    unsigned long nr, block_size;
\tfd = open(name, O_RDONLY);
\tif (0 == ioctl(fd, HDIO_GETGEO, &hdg)) 
\t\tprintf(\"Geometry: cylinders=%d, heads=%d, sectors=%d\\n\", hdg.cylinders, hdg.heads, hdg.sectors);

\tint size = request2(\"Block get size\", BLKGETSIZE);
\trequest2(\"Block get size 64\", BLKGETSIZE64);\t
\trequest2(\"Block physical get\", BLKPBSZGET);
\tint logical = request2(\"Block logical get\", BLKSSZGET);\t
\tprintf(\"Calculated size: %lu\\n\", size * logical);

\treturn 0;

}
<\/pre>\n
{\/codecitation}<\/p>\n
Conclusion<\/h2>\n
I still have to perform additional tests with the WD hard drive. So far, I’ve narrow down the problem to the scsi command READ CAPACITY received from the hard drive that is incorrect. I’m not able to determine if READ CAPACITY(16) is missing from the hard drive firmware, or if the USB dock station prevent me from using this command. Also, I’m not able to fully understand why the kernel driver don’t have workaround. Although it’s using the scsi command to determine the capacity, ATA identify command are executed (observed in my USB monitoring) later on. Therefore, the kernel driver at some point should be able to retrieve the correct information with regards to the real block sizes. I’m just not completely satisfied to observe that the disk still works with Windows, and I’m unable to have a valid alternative in my linux environment. Any hints are welcome \ud83d\ude09<\/p>\n
A big thank you for the blog entry in [1]. I fall by serendipity on it and it was really helpful as I’ve started understanding the real issue from this post. Other resources are worth to read in [2] and [3]. The messages in [4] discuss about similar issues.<\/p>\n
[1] http:\/\/johannes-bauer.com\/linux\/wdc\/?menuid=3<\/a>
[2] https:\/\/wiki.archlinux.org\/index.php\/Advanced_Format<\/a>
[3] http:\/\/people.redhat.com\/msnitzer\/docs\/linux-advanced-storage-6.0.pdf<\/a>
[4] https:\/\/lkml.org\/lkml\/2011\/9\/26\/324<\/a>
[5] http:\/\/sg.danny.cz\/sg\/sg3_utils.html
<\/a>[6] https:\/\/wiki.ubuntu.com\/Kernel\/Debugging\/USB<\/a>
[7] http:\/\/community.wd.com\/t5\/Desktop-Mobile-Drives\/3TB-Internal-Drive-as-746GB\/td-p\/356135<\/a>
[8] https:\/\/en.wikipedia.org\/wiki\/SCSI_command<\/a>
[9] http:\/\/codingrelic.geekhold.com\/2012\/02\/ata-commands-in-python.html<\/a>
[10] https:\/\/gist.github.com\/shimarin\/34f05caaf75d02966a30<\/a><\/p>\n
{jcomments on}<\/p>\n
\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"
I’ve been struggling for quiet a long time about a problem with a Western Digital 3TB hard drive. I let it aside as, back in time, I found a quick workaround which is not efficient on the long term: using windows to partition the disk, and use windows to read\/write on it. I’ve finally decided … Continuer la lecture de « Too hard to drive Advanced Format »<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15],"tags":[],"class_list":["post-115","post","type-post","status-publish","format-standard","hentry","category-astuces"],"_links":{"self":[{"href":"https:\/\/nunix.fr\/index.php?rest_route=\/wp\/v2\/posts\/115","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nunix.fr\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nunix.fr\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nunix.fr\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nunix.fr\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=115"}],"version-history":[{"count":0,"href":"https:\/\/nunix.fr\/index.php?rest_route=\/wp\/v2\/posts\/115\/revisions"}],"wp:attachment":[{"href":"https:\/\/nunix.fr\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=115"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nunix.fr\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=115"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nunix.fr\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=115"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}