QUICKLY copying files from one NAS to another using wget [ Edited ]

[Ron_KK1L]

Folks,

I thought I would share a fast way to copy files from one NAS to another. I had the misfortune of my MBL 2TB drive starting to fail. WD made good by replacing it. I needed to copy my data to the new drive so I could send the failing drive back to WD. Doing this through Windows file copy or FreeFileSync or some other way was going to take a long time. A small bit of research revealed that "wget" could do the job. "wget" is already installed on the MBL, so NO HACKING is required! GREAT NEWS.

So here is what I did to copy all the files from "Public" on one MBL to "Public" on another:

(Updated switches 12-FEB-2012 to fully copy drive)

1) Enable ssh on your MBL (http://ip-address-of-MBL/UI/ssh)

2) Log onto the MBL with putty or something similar

3) cd /shares (this puts you at the root of the Windows, etc accessible files. You should see "Public" here.)

4) wget -mirror  -np -x --no-host-directories -o "directory/logfilename" ftp://adminassword@ip-address/Public

Replace "password" with the admin password for the MBL

Replace "ip-address" witht the ip-address of the NAS you would like to copy FROM

"-mirror" means recursively drill down unlimited depth

"-np" keeps recursion below the parent (probalby not needed)

"-x" creates even empty directories if they exist

"--no-host-directories" means do not prepend the ip-address to the local directory structure

On my 1GB network moving files from a 2TB MBL to a 6TB MBL-Duo went at a rate of 20-35MB/s! Just awesome.

For example to transfer a 21GB directory took 21min. I had all 705GB of data copied in...well the MBL I am taking the data from is failing, so it does not make it through all 705GB before I have to unplug/replug the drive. The average transfer rate over all files copied (small, large, and otherwise) is 19MB/s for me.

Note: is "colon" followed by "p" in the wget command.

[Vadir]

Very helpful information, thanks for sharing it.

[Hetty]

Hello, 

I have the same problem of trying to transfer all my data from one defective MybookLive to another. It will take 6 full days to transfer if i  simply copy and paste. Transfer rate is 1MB/sec 

My defective drive folders can be accessed but the dashboard can not.

Could you let me know the best and fastest way to do this please.

With kind regards 

Urgent Hetty from the Netherlands 

[Ron_KK1L]

Are you able to log on through ssh (secure shell)? If you install putty (search with google) on your computer you can do this.

[Hetty]

Hi Ron, 

I have enabled ssh on my new MBL and I have also downloaded putty. But when I come to use putty, it opens a black page and asks 'sign on as 'I type admin and then it asks for a password but with a little yellow curser that won't allow any input. My mbl has no password at this time.

Entering simply causes a bad password response

My sytem is 64bit win7 ultimate but putty is x86..is this the problem or something else.

I have rebooted both  drives and pc.

Hetty

[Ron_KK1L]

Ah....the user id is "root" and the password is the same password you have assigned for web interface access. You are logging onto a linux machine. "root" is the equivalent of administrator. Do you know your way around linux at all? The command to change directories is that same as for Windows command window (DOS for the old timers) "cd". However to list files the command is "ls" and not "dir".

[Hetty]

Hi Ron, 

'cd /shares (this puts you at the root of the Windows, etc accessible files. You should see "Public" here.) '

Having input cd /shares I do not see Public.

It is true that I am not at all familiar with Linux.

[Hetty]

Ok. 

when I type cd / shraes i get nothing

when I type: ls / shares I get

line 1: Public

line 2: MyBookLive: / shares#

I then input:

wget -mirror -np -x --no-host-directories -o "directory/logfilename" ftp://admin:yyyyyyyyyyyyy@192.168.2.1/Public

yyy's are my password

I then get: wget: directory/logfilename: No such file or directory

What am I doing wrong?

Hetty

[Ron_KK1L]

Ah!! You are SOOOOO close!

The output you shared is exactly what I would expect. Instead of typing "ls /shares" you would do better to just type "ls". This way you will list the CURRENT directory to make sure you are in the right one. You can also type "pwd" and the current directory name will be shown.

I have confused you by adding the option to have the results of the command be directed to a log file. The value "directory/logfile" is meant for you to enter the path and filename for where you would like to have the ouput of the wget command go to. So...let's skip that step.

Use this command:

wget -mirror -np -x --no-host-directories ftp://admin:yyyyyyyyyyyyy@192.168.2.1/Public

What you will see now is LOTS of activity on the screen. Something like this....

--2012-02-12 09:46:45--  ftp://admin:*password*@10.0.0.135/Public/Shared%20Pictures
           => `Public/.listing'
Connecting to 10.0.0.135:21... connected.
Logging in as admin ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD /Public ... done.
==> PASV ... done.    ==> LIST ... done.

     0K                                                       20.1K=0.2s

2012-02-12 09:46:46 (20.1 KB/s) - `Public/.listing' saved [3573]

--2012-02-12 09:46:46--  ftp://admin:*password*@10.0.0.135/Public/Shared%20Pictures/Shared%20Pictures
           => `Public/Shared Pictures/.listing'
==> CWD /Public/Shared Pictures ... done.
==> PASV ... done.    ==> LIST ... done.

     0K                                                        203K=0.02s

2012-02-12 09:46:46 (203 KB/s) - `Public/Shared Pictures/.listing' saved [3767]

09:47:32--  ftp://admin:*password*@10.0.0.135/Public/Shared%20Pictures/FromOlympusArchive/2007/WallProgress/Originals/Shared%20Pictures
 `Public/Shared Pictures/FromOlympusArchive/2007/WallProgress/Originals/.listing'
ic/Shared Pictures/FromOlympusArchive/2007/WallProgress/Originals ... done.
done.    ==> LIST ... done.
                                                  826K=0.001s
:47:32 (826 KB/s) - `Public/Shared Pictures/FromOlympusArchive/2007/WallProgress/Originals/.listing' saved [604]
09:47:32--  ftp://admin:*password*@10.0.0.135/Public/Shared%20Pictures/FromOlympusArchive/2007/WallProgress/Originals/P7060004.JPG
 `Public/Shared Pictures/FromOlympusArchive/2007/WallProgress/Originals/P7060004.JPG'
equired.
done.    ==> RETR P7060004.JPG ... done.
4 (596K)
... .                                           100% 18.1M=0.03s
:47:32 (18.1 MB/s) - `Public/Shared Pictures/FromOlympusArchive/2007/WallProgress/Originals/P7060004.JPG' saved [609994]
09:47:32--  ftp://admin:*password*@10.0.0.135/Public/Shared%20Pictures/FromOlympusArchive/2007/WallProgress/Originals/P7060005.JPG
 `Public/Shared Pictures/FromOlympusArchive/2007/WallProgress/Originals/P7060005.JPG'
equired.
done.    ==> RETR P7060005.JPG ... done.
2 (609K)
... .                                           100% 31.1M=0.02s

[Hetty]

Hi Ron, 

Thank you for your great explanation. Everything worked up to the point where my login failed.

I have reset my Mybooklive (defective one) and assumed that admn:admin would work but no. I also tried my old passwords and 123456 but everything fails..any thoughts.

Here is what i do and the result:

login as: root
root@192.168.2.2's password:
Linux MyBookLive 2.6.32.11-svn48181 #1 Thu Sep 15 18:22:06 PDT 2011 ppc

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.

Disclaimer: SSH provides access to the network device and all its
content, only users with advanced computer networking and Linux experience
should enable it. Failure to understand the Linux command line interface
can result in rendering your network device inoperable, as well as allowing
unauthorized users access to your network. If you enable SSH, do not share
the root password with anyone you do not want to have direct access to all
the content on your network device.

MyBookLive:~# ls /shares
Public wget-log
MyBookLive:~# wget -mirror -np -x --no-host-directories ftp://admin:admin@192.168.2.1/Public
--2012-02-15 23:15:45-- ftp://admin:*password*@192.168.2.1/Public
=> `.listing'
Connecting to 192.168.2.1:21... connected.
Logging in as admin ...
Login incorrect.
rror: No such file or directory
No URLs found in rror.
MyBookLive:~#