WD SHARESPACE RAID 5 FAILURE [ Edited ]

[Cyberblitz]

Hello all...

Ok, i know this subject has been discussed before, including a nice HOW-TO by dudemanbubba: (http://community.wdc.com/t5/WD-ShareSpace/HOWTO-Sharespace-RAID-5-Data-Recovery/td-p/287736) but my situation is a little differen. I dont' have access to a desktop computer, only a laptop,  therefore can't insert a PCI Multi-Sata Card.

Anyways, my situation: my wife plugged in a kettlle and powered it on. For some reason it tripped the whole house, switching everything off, inclduing the NAS drive. When we discovered it was the problem kettle, we swtiched everything back on and I worryingly ran to look at the NAS drive, which showed 4 failed drives. I did the normal things, switched it off and truned it back on...the same... I accessed the GUI interface which displayed the drives as failed, giving me the option to remove or format. At first I thought the drives where buggered but after reading around I realised it was the RAID controller not the drives which had failed.

I accessed the NAS drive using a Gentoo LIVECD, utilising SSH.

ssh root@"your NAS IP address" --- (withouth the parenthesis)

password: "welc0me" ---  (without the parenthesis)

     YOU WILL THEN BE IN THE "SSH" SHELL

I succesfully logged into the system and typed 'fdisk -l' and to my delight could see all the drives pop up with their relvent partitions. That supported the idea the drives were fine. For some reason though, WD format each drive with 4 partitons, the last holding all the data and which the chosen RAID (5 in my case) is applied. After talking around with a few people, it seems WD utilise the hard drives for a very basic UNIX/LINUX base system (hence the other 3 partitions), suggesting the RAID configuration is software based. This make things a little complicated.

Next, I examined the drives using "cat /proc/mdstat" and "mdadm --examine --scan" revealing that the drives were good too (I don't have the outputs of these, sorry). I don't remeber eveything about the details but the outputs showed the drives were clean, not degraded. It surprises me that I can manage to access the system this way but the system maintains the drives have failed!? but they obviously have not.

Anyways, I deceide to be safe (and because i thiought i didn't have the relevent hardware) to take to an IT store. They didn't do data recovery per se but did have some success in RAID recovery, and they were cheap. I also made sure they cloned the drives before their recovery attempt, which they did normally anyway. After a week of them messing around, they stated they could not access the GUI interface nor see their partitions. This got me worried they'd mucked it all up. But, alas, they did not. When I got it back home, it was still in the same state as I sent it.. I could till access and  see the drives and their partitions and access the GUI interface.. Why could they not???? I reset the GUI interface so it would be defaulted for easy access for them... I got further than they did in a few hours to their week...  Not taking things their again.. Useless..

Now I have it home, i'm reattempting recovery.. I bought an addittional 2TB drive to 2TB I already had and have them both in individual USB caddies. I did thing about using DD to clone the relevant partions to the other drives but have been pursuaded to use ddrescue instead. DD will stop if it finds any bad blocks, ddrescue does not.

This is how I have set up things cuurently:

I have one of the 1TB drives from the NAS drive housed in one of the USB caddies and a 2TB drive in the other, which has 2 partitions on it. I've initated the ddrescue to clone the 4th partition of the NAS drive to the 1st partion of the 2TB drive.

ddrescue -v  -f  /dev/sdb4 /dev/sdc1 bs=64k

(-v = verboses (displays text of whats going on) the porcess

-f = forces the process to overwrite the destination

/dev/sda4 (the 4th partition of the NAS drive (holding all the data you need) BE AWARE: THIS CAN CHANGE DEPENDING ON THE ORDER YOUR SYSTEM REGISTERES THE HARD DRIVES. IN MY CASE, I HAVE AN INTERNAL HARD DRIVE (dev/sda) AND TWO USB DRIVES, ONE HOLDING THE NAS DRIVE (/dev/sdb), THE OTHER IS THE DESTINATION DRIVE (/dev/sdc).

/dev/sdc1 = 1st partition of the destination hard drive.

bs=64k = block size is 64k (this was determined by examining the NAS using 'mdadm --examine /dev/sdb4' - however, 64k is pretty standard from what I understand)

This is where i'm upto. I'm currently cloning the 4th partition of the NAS drives into 4 partitions on 2 2TB drives (ensuring to maintain the order of the NAS drives as they are in the NAS housing).This ensures the original drives don't become damaged by mistakes i may make.

I will then attempt to reassemble the RAID using mdadm.

My only concern regarding this method is I'm sure not it will work, whether i can reassmeble the RAID with 2 TB with 2 partitions on each.. I've searched the interent for an answer and by asking around but I haven't found an answer. Is this possible, i'm assuming it can at this point...

Does anyone have any suggestions???

[Alucardx23]

Don't know if you will be able to recover your files from the two 2TB hard drives, if I were you I would have gone with the dudemanbubba guide, seems like a safer bet.

[dudemanbubba]

You are pioneering new territory here.  I have no idea if what you are doing will work.  I am somewhat savvy with computers in terms of being able to tinker and keep things running.  But I don't really understand the underlying technology of how the things work on the software level.

My two cents... the money spent on the additional 2TB drive could have bought you a machine at a garage sale with enough in it to do the job.  If you have data that you treasure, I would tread lightly in the direction you are going.

The fact you can get into ssh may be a good sign as you can see the unit is functioning.  I know it has a stripped down linux on it.  Perhaps there is enough on there to "send" your data to a remote location?  Is ftp available from the command line on the SS when logged into ssh?  I don't have my unit any more or I would check myself...

Good luck to you!  I hope you get things resolved.  Post back if you do for others to share!

[Cyberblitz]

Thanks for your replies..

The problem with following dudemanbubba method is I wasn't keen on spending an a destop computer and new drives as well. The drives I have are more like an investment to assembling a larger RAID later on and I will also have back-up drives on top. 

I agree my method is a bit untested, considering no mention of this method been done any where on the net I can find.. I'm just hoping mdadm will consider each partition as seperate entities, as it does in dudemanbubba method, but having two parts of the RAID on each of the two drives... We'll see.. 

I'm in the middle of still cloning the drives so I can mess with the cloned RAID parts instead of messing up the originals if i make a mistake. But the cloning is painfully slow, running at 44MB/s. And this is using a 2 Sata HDD dock via a eSata cable. I changed to this method from the two USB caddies which were running at half this. I'm not sure how to speed this up. Oh, and I deceided to use DD now too.

[Cyberblitz]

EXCITEMENT...

I have just found evidence to support that you can infact indeed partition a drive and have each partition as part of the array. This is generally not done because if that drive failed, the RAID is buggered and unrecoverable (example: you have 4 drives, 3 drives are 1TB and the forth is 2TB but split into 2, gving you 5 parts to your array. Howver, if your 2TB drive failed, your array is buggered). However, you can't do this in the WD Sharespace because all drives have to be the same size and there is no option to partiton them..

But you can if you built you array in a Linux environment.  So there is light at the end of the tunnel.

[Cyberblitz]

Holy Moly..

This is far more complicated than I thought... I can not for the life of me get into these disks... How is it possible for this to happen so seriously.. We rely on these products to do the job their designed to do,  too much... I'm exhughsted and really depressed that i may have lost all my precious files, including the birth of my children. WD are completly useless to deal with who have no meanignful solutions to this re-occuring problem, for many, many people... There should be some safty device installed in the device to stop this.. I think my particular problems stems from the a power cut not only nonce but twice.. I noticed that the WD sharespace automatically switches on when power is restored, and i think this is the problem... Why does this machine power on by its self??? Even when i plug the power cord in the bcak it turns on without me pressing the power button... rediculas.. And becasue of this, is buggered... Thanks WD for nothing!!!

[dudemanbubba]

Have you modified the original disks at all or have you tried to make duplicates?  If you still have the original disks in tact then your data should be there.  Remember, you only need three of the four disks to make this work since it is a RAID 5.   If you modified partitions on the original disks at all you are probably out of luck.

Perhaps you know someone with an old pc that you could use for the purpose.  With my tutorial, you will not affect the original machine at all.  You will be unplugging their drives and plugging yours in temporarily.  Once you are done you can plug the original drives back in and all should be good to go.

Sorry you are having so much trouble...

[Cyberblitz]

Hey, No, havn't modified them.... I believe. I have accessed the SSH shell gving me access to the disks and implemeted LVM2 protocols. I beleive this won't change anything..i hope... I can still see all the partitions but there is no LV group or VG group, which I'm not sure what this means... I really could do with a 4 bay caddie and sticking them in. But i don't.. I don't know anyone who would borrow me their computer.. I'm really stuck on this on... I managed to clone the drives as described earlier, but there was no superblock for me to do anything with (although i could view the detail of the RAID drives, with superblock being consitant)... No FS to be recognised.. But my setup could be the problem with this... Well, i;m currently creating a RAID 5 on my spare 2TB drives which have 2 paritions on each (making it a four part RAID), then i'll LVM it. Then i'll clone the drives on to the four parts.. I don't know if this will work, but least there will be base FS system beforehand...

[Cyberblitz]

Well, this is what i'm able to ascertain from my device:

(NOTE: i have removed one of the disks from this array for the moment)

I could also not use fsck.ext2 either.. Would not do anything, just stated I had missing superblocks...

~ $ fdisk -l

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sda1               1          26      208844+  fd  Linux raid autodetect
/dev/sda2              27         156     1044225   fd  Linux raid autodetect
/dev/sda3             157         182      208845   fd  Linux raid autodetect
/dev/sda4             183      121601   975298117+  fd  Linux raid autodetect

Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdb1               1          26      208844+  fd  Linux raid autodetect
/dev/sdb2              27         156     1044225   fd  Linux raid autodetect
/dev/sdb3             157         182      208845   fd  Linux raid autodetect
/dev/sdb4             183      121601   975298117+  fd  Linux raid autodetect

Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdc1               1          26      208844+  fd  Linux raid autodetect
/dev/sdc2              27         156     1044225   fd  Linux raid autodetect
/dev/sdc3             157         182      208845   fd  Linux raid autodetect
/dev/sdc4             183      121601   975298117+  fd  Linux raid autodetect
~ $ mdadm --detail /dev/sd[abc]4
mdadm: /dev/sda4 does not appear to be an md device
mdadm: /dev/sdb4 does not appear to be an md device
mdadm: /dev/sdc4 does not appear to be an md device
~ $ cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid5] 
md124 : active raid1 sda2[3]
      1044160 blocks [4/1] [___U]
      
md1 : active raid1 sdc2[0] sdb2[2]
      1044160 blocks [4/2] [U_U_]
      
md0 : active raid1 sdc1[1] sdb1[3] sda1[2]
      208768 blocks [4/3] [_UUU]
      
unused devices: <none>
~ $ mdadm --examine --scan
ARRAY /dev/md0 level=raid1 num-devices=4 UUID=ff74d9bf:5ab0ca74:3301d4f1:44541c6d
ARRAY /dev/md1 level=raid1 num-devices=4 UUID=44c458bb:e1ebf9e8:1cd440a8:86a64cde
ARRAY /dev/md2 level=raid5 num-devices=4 UUID=3049bb1e:078f8250:46c2159e:49a0be43
~ $ mdadm --examine /dev/sd[abc]4
/dev/sda4:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 3049bb1e:078f8250:46c2159e:49a0be43
  Creation Time : Thu Oct 28 07:37:28 2010
     Raid Level : raid5
  Used Dev Size : 975097920 (929.93 GiB 998.50 GB)
     Array Size : 2925293760 (2789.78 GiB 2995.50 GB)
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 2

    Update Time : Sun Feb 19 18:07:33 2012
          State : clean
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 47bf1668 - correct
         Events : 1527202

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     0       8        4        0      active sync   /dev/sda4

   0     0       8        4        0      active sync   /dev/sda4
   1     1       8       20        1      active sync   /dev/sdb4
   2     2       8       36        2      active sync   /dev/sdc4
   3     3       8       52        3      active sync   /dev/sdd4
/dev/sdb4:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 3049bb1e:078f8250:46c2159e:49a0be43
  Creation Time : Thu Oct 28 07:37:28 2010
     Raid Level : raid5
  Used Dev Size : 975097920 (929.93 GiB 998.50 GB)
     Array Size : 2925293760 (2789.78 GiB 2995.50 GB)
   Raid Devices : 4
  Total Devices : 3
Preferred Minor : 2

    Update Time : Wed Mar  7 12:51:38 2012
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 47d5360c - correct
         Events : 1527223

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     1       8       20        1      active sync   /dev/sdb4

   0     0       0        0        0      removed
   1     1       8       20        1      active sync   /dev/sdb4
   2     2       8       36        2      active sync   /dev/sdc4
   3     3       8       52        3      active sync   /dev/sdd4
/dev/sdc4:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 3049bb1e:078f8250:46c2159e:49a0be43
  Creation Time : Thu Oct 28 07:37:28 2010
     Raid Level : raid5
  Used Dev Size : 975097920 (929.93 GiB 998.50 GB)
     Array Size : 2925293760 (2789.78 GiB 2995.50 GB)
   Raid Devices : 4
  Total Devices : 3
Preferred Minor : 2

    Update Time : Wed Mar  7 12:51:38 2012
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 47d5361e - correct
         Events : 1527223

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     2       8       36        2      active sync   /dev/sdc4

   0     0       0        0        0      removed
   1     1       8       20        1      active sync   /dev/sdb4
   2     2       8       36        2      active sync   /dev/sdc4
   3     3       8       52        3      active sync   /dev/sdd4
~ $ pvs 
  /dev/sdd: open failed: No such device or address
  /dev/sdd3: open failed: No such device or address
~ $ pvscan
  No matching physical volumes found
~ $ lvscan
  No volume groups found

[Cyberblitz]

Ok, here is where I'm at with this problem... I have cloned all my drives and reconstructed a RAID 5 with new drives so that I could explore a working system.. And here is what I've discovered: I can not find any Logical Volume Managment System (LVM) in place; lvscan, pvscan, vgscan reveal there is nothing in place... Even typing lvmdiskscan ouputs that there no LVM structure in place.. Typing 'mount' would display the mounted LVM, but there isn't... If from reading in these forums about an LVM being implemented on ShareSpace then why isn't it there on a working system???? I get the feeling this may be region based or some systems have it implemented and some don't... Mine doesn't... I have the latest firmware so it isn't that either... I thought I would try 'data quoting' a user to see if that creates an LVM but it doesn't... The RAID 5 structure is mounted onto /dev/md2 - this holds all your data... However, /dev/md2 is mounted in several different directories I.e. /DataVolume, /shares/Public, /shares/Download (remember, this is how it is at default, you may have more depending on how many share folders you have created)... But, what's confusing is that the share folders are not just in the shares directory but also in the DataVolume directory: /DataVolume/Public, /DataVolume/Downloads, /DataVolume/_torrent_ (but these file paths are not mounted, only the parent directory is mounted, /DataVolume (note: _torrent_ isn't in the shares folder)).... There is also a folder in the /DataVolume directory called 'jewab'' I have no idea what this folder is about, but it does contain some Twonkymedia stuff. There is a also a folder of the same name in the root directory but it contains nothing... /DataVolume is also where the Data Quota files are kept... I realise a lot people, including myself, have had DataVolume doesn't exist problems.. Unfortunately, I can't explore this any more on my cloned drives.. However, I,m hoping i can still retrieve my data from the 4th partition, now I have a better understanding of the file structure built on my ShareSpace... I also know now that the file system is ext3 too... And also there is no md superblock on /dev/md2.. I hope this helps a little for those of you in the same predicament, and if anyone can shed more light on what I've described here, that would be awesome... I'll update more following my attempts at retrieving my data...