Backing up Windows computers to a Synology NAS via SSH and rsync

January 4th, 2012
Share

I recently purchased a Synology DS1511+ to act as a NAS (network attached storage) for my home network. The 5-drive, Linux powered device is beautiful – small, sleek and quiet. What sold me was the amazing web-based configuration interface they provide, and the ability to access the device remotely via the web or from mobile apps Synology provides in the iTunes App Store and Android Market.

After setting it up with a couple 2TB and 3TB drives, I wanted to use the device to backup documents from several Windows computers I manage (my own, my wife’s netbook and my parents’ computers thousands of miles away). Local network backup is pretty easy – you can use the Synology Data Replicator to backup Windows hosts to your Synology on your local network. However, it seemed pretty slow to me, and doesn’t use the highly-optimized rsync protocol for backing up files. Since I was previously using rsync over SSH to a Linux server I run at home, I figured since the Synology was Linux-based, it should be able to do the same.

All it takes is a few updates to the Synology server, and a few scripts on the Windows computers you want to backup to make this work for both computers on your home network as well as any external computers you want to backup, as long as they know the address of the remote server. You can use a dynamic-IP service such as TZO.com or DynDNS.org so your remote Windows clients know how to contact your home Synology.

Once I got it all working, I figured the process and scripts I created could be used by others with a Synology NAS (or any server or NAS running Linux). I’ve created a GitHub repository with the scripts and instructions so you can setup your own secure backup for local and remote Windows computers:

https://github.com/nicjansma/synology-windows-ssh-rsync-backup

Features

  • Uses rsync over ssh to securely backup your Windows hosts to a Synology NAS.
  • Each Windows host gets a unique SSH private/public key that can be revoked at any time on the server.
  • The server limits the SSH private/public keys so they can only run rsync, and can’t be used to log into the server.
  • The server also limits the SSH private/public keys to a valid path prefix, so rsync can’t destroy other parts of the file system.
  • Windows hosts can backup to the Synology NAS if they’re on the local network or on a remote network, as long as the outside IP/port are known.

NOTE: The backups are performed via the Synology root user’s credentials, to simplify permissions. The SSH keys are only valid for rsync, and are limited to the path prefix you specify. You could change the scripts to backup as another user if you want (config.csv).

Synology NAS Setup

  1. Enable SSH on your Synology NAS if you haven’t already. Go to Control Panel – Terminal, and check “Enable SSH service”.
  2. Log into your Synology via SSH.
  3. Create a /root/.ssh directory if it doesn’t already exist
    mkdir /root/.ssh
    chmod 700 /root/.ssh
  4. Upload server/validate-rsync.sh to your /root/.ssh/validate-rsync.sh. Then chmod it so it can be run:
    chmod 755 /root/.ssh/validate-rsync.sh
  5. Create an authorized_keys file for later use:
    touch /root/.ssh/authorized_keys
    chmod 600 /root/.ssh/authorized_keys
  6. Ensure private/public key logins are enabled in /etc/ssh/sshd_config.
    vi /etc/ssh/sshd_config

    You want to ensure the following lines are uncommented:

    PubkeyAuthentication yes
    AuthorizedKeysFile .ssh/authorized_keys
  7. You should reboot your Synology to ensure the settings are applied:
    reboot
  8. Setup a share on your Synology NAS for backups (eg, ‘backup’).

Client Package Preparation

Before you backup any clients, you will need to make a couple changes to the files in the client/ directory.

  1. First, you’ll need a few binaries (rsync, ssh, chmod, ssh-keygen) on your system to facilitate the ssh/rsync transfer. Cygwin can be used to accomplish this. You can easily install Cygwin from http://www.cygwin.com/. After installing, pluck a couple files from the bin/ folder and put them into the client/ directory. The binaries you need are:
    chmod.exe
    rsync.exe
    ssh.exe
    ssh-keygen.exe

    You may also need a couple libraries to ensure those binaries run:

    cygcrypto-0.9.8.dll
    cyggcc_s-1.dll
    cygiconv-2.dll
    cygintl-8.dll
    cygpopt-0.dll
    cygspp-0.dll
    cygwin1.dll
    cygz.dll
  2. Next, you should update config.csv for your needs:
    rsyncServerRemote - The address clients can connect to when remote (eg, a dynamic IP host)
    rsyncPortRemote - The port clients connect to when remote (eg, 22)
    rsyncServerHome - The address clients can connect to when on the local network (for example, 192.168.0.2)
    rsyncPortHome - The port clients connect to when on the local network (eg, 22)
    rsyncUser - The Synology user to backup as (eg, root)
    rsyncRootPath - The root path to back up to (eg, /volume1/backup)
    vcsUpdateCmd - If set, the version control system command to use prior to backup up (eg, svn up)
  3. The version control update command (%vcsUpdateCmd%) can be set to run a version control update on your files prior to backing up. This can be useful if you have a VCS repository that clients can connect to. It allows you to make remote changes to the backup scripts, and have the clients get the updated scripts without you having to log into them. The scripts are updated each time start-backup.cmd is run. For example, you could use this command to update from a svn repository:
    vcsUpdateCmd,svn up

    If you are using a VCS system, you should ensure you have the proper command-line .exes and .dlls in the client/ directory. I’ve used Collab.net’s svn.exe and lib*.dll files from their distribution (http://www.collab.net/downloads/subversion/).

    During client setup, you simply need to log into the machine, checkout the repository, and setup a scheduled task to do the backups (see below). Each time a backup is run, the client will update its backup scripts first.

The client package is now setup! If you’re using %vcsUpdateCmd%, you can check the client/ directory into your remote repository.

Client Setup

For each client you want to backup, you will need to do the following:

  1. Generate a private/public key pair for the computer. You can do this by running ssh-keygen.exe, or have generate-client-keys.cmd do it for you:
    generate-client-keys.cmd

    or

    generate-client-keys.cmd [computername]

    If you run ssh-keygen.exe on your own, you should name the files rsync-keys-[computername]:

    ssh-keygen.exe -t dsa -f rsync-keys-[computername]

    If you run ssh-keygen.exe on your own, do not specify a password, or clients will need to enter it every time they backup.

  2. Grab the public key out of rsync-keys-[computername].pub, and put it into your Synology backup user’s .ssh/authorized_keys:
    vi ~/.ssh/authorized_keys

    You will want to prefix the authorized key with your validation command. It should look something like this

    command="[validate-rsync.sh location] [backup volume root]" [contents of rsync-keys-x.pub]

    For example:

    command="/root/.ssh/validate-rsync.sh /volume1/backup/MYCOMPUTER" ssh-dss AAAdsadasds...

    This ensures that the public/private key is only used for rsync (and can’t be used as a shell login), and that the rsync starts at the specified root path and no higher (so it can’t destroy the rest of the filesystem).

  3. Copy backup-TEMPLATE.cmd to backup-[computername].cmd
  4. Edit the backup-[computername].cmd file to ensure %rsyncPath% is correct. The following DOS environment variable is available to you, which is set in config.csv:
    %rsyncRootPath% - Remote root rsync path

    You should set rsyncPath to the root remote rsync path you want to use. For example:

    set rsyncPath=%rsyncRootPath%/%COMPUTERNAME%

    or

    set rsyncPath=%rsyncRootPath%/bob/%COMPUTERNAME%

    %rsyncRootPath% is set in config.csv to your Synology backup volume (eg, /volume1/backup), so %rsyncPath% would evaluate to this if your current computer’s name is MYCOMPUTER:

    /volume1/backup/MYCOMPUTER

    You can see this is the same path that you put in the authorized_keys file.

  5. Edit the backup-[computername].cmd file to run the appropriate rsync commands. The following DOS environment variables are available to you, which are set in start-backup.cmd:
    %rsyncStandardOpts% - Standard rsync command-line options
    %rsyncConnectionString% - Rsync connection string

    For example:

    set cmdArgs=rsync %rsyncStandardOpts% "/cygdrive/c/users/bob/documents/" %rsyncConnectionString%:%rsyncPath%/documents
    echo Starting %cmdArgs%
    call %cmdArgs%
  6. Copy the client/ directories to the target computer, say C:\backup. If you are using %vcsUpdateCmd%, you can checkout the client directory so you can push remote updates (see above).
  7. Setup a scheduled task (via Windows Task Scheduler) to run start-backup.cmd as often as you wish.
  8. Create the computer’s backup directory on your Synology NAS:
    mkdir /volume1/backup/MYCOMPUTER

The client is now setup!

Source

As noted above, the source for these scripts is available on Github:

https://github.com/nicjansma/synology-windows-ssh-rsync-backup

If you have any suggestions, find a bug or want to make contributions, please head over to GitHub!

  1. April 7th, 2012 at 11:08 | #1

    it is a bit hard for non experienced user to do this !

    is it possible to save ours computers to a remote DS in another way ?

    rsync, webdav ? or anything else ?

    thanks for sharing !

    Sebastien

    • April 8th, 2012 at 19:27 | #2

      Hi Sebastien,

      Yes, you should be able to simply enable rsync on the Synology and open the appropriate port on your firewall/router to forward it. This method uses the same thing, however, it encrypts the transmission over SSH.

  2. May 7th, 2012 at 03:46 | #3

    Nic,

    Great post!

    One potential issue with this approach is that the backup data is stored in the clear on the DS. It would be interesting to consider ways to encrypt the backup with keys known to the client only.

    .. WebDAV with a TrueCrypt virtual disk .. Not sure what performance would be like.

    Steve

    • May 7th, 2012 at 11:55 | #4

      I agree completely, and I’ve been pondering what the solution would be. It’d be nice to have the clients have their own keys that they could use to mount a filesystem (or a .TC filesystem file) before the rsync, and then unmount afterward. There’s always a trade-off between full-automation and security. Anything that is fully automated still has the possibility of being infiltrated and then that system could be used to infiltrate further.

      Let me know if you come up with anything!

  3. Andreas
    June 19th, 2012 at 11:17 | #5

    @ Steve Upp

    You could use the built in encryption of the synology. When the share is mounted it will work as a normal folder in ssh.

  4. Grant
    November 5th, 2012 at 14:45 | #6

    Great post. Looks like just what I need if I could get it to work 🙂
    I think I have it all set up correctly, but when I run the start-backup.cmd it fails
    with a “Host key verification failed.” message.
    If I run the ssh command on its own. it works fine:
    eg.
    ssh -i rsync-keys-mymedia -l root pwd
    with the validate-rsync-.sh script only containing the line
    $SSH_ORIGINAL_COMMAND
    for testing.
    Any idea what I could be doing wrong.
    Note running alll this from a cygwin cmd shell.

  5. Grant
    November 5th, 2012 at 14:56 | #7

    @ Grant

    Never mind…fixed it 🙂
    Found a typo.
    Thanks anyway.

  6. November 5th, 2012 at 15:26 | #8

    Glad it worked! Was it a typo in your script or in mine?

  7. mac
    December 5th, 2012 at 13:21 | #9

    hi nic

    i am trying to get this mentioned dll file “cygspp-0.dll”. but have a real hard time to find it.
    could you give advice, helping to locate it?
    all the other files where as expected found after installation of the current cygwin setup.

    kind regards
    markus

    • December 5th, 2012 at 15:06 | #10

      Hi Markus,

      Newer versions of Cygwin may not need that DLL. Does the backup occurr successfully or is it complaining about cygspp-0.dll being missing?

  8. Mac
    December 5th, 2012 at 22:28 | #11

    hi nic
    actually i am not that far. the problem occurs when executing the client script: generate-client-keys.cmd
    thanks for advice.
    best regards
    markus

  9. BillB
    December 10th, 2012 at 09:24 | #12

    start-backup.cmd keeps prompting for password.
    In authorized_keys it says ssh-dss. But generate-client-keys has ssh-keygen -t rsa. Should these be the same? When I changed ssh-dss to ssh-rsa it ran without passwd, but then copied files were unreadable.
    Thanks for the post.
    BB

  10. December 15th, 2012 at 12:07 | #13

    @Mac
    I just did a new install of Cygwin. I selected two additional packages besides the default: ‘rsync’ and ‘openssh’. The openssh package installed cygssp.dll into the cygwin\bin\ dir.

  11. December 15th, 2012 at 12:10 | #14

    @BillB
    I would use whatever comes from the .PUB file. Does your server give you more information about why it rejects the ssh-dss key?

  12. Rene
    January 5th, 2013 at 09:06 | #15

    Hi,
    I did not want to run anything special on the home PC’s and I am also sure that I will forget to run a backup on those PC’s. So I made this the other way around: The Syno is “pulling” the data from the PC’s by first mounting drives or folders from the PC to the Syno.
    All works fine except one thing. If I have a file name with special characters, like in my name (René) the backup file gets a kind of underscore and is not accessible via windows explorer. I tried to add –iconv=utf-8,ISO88592 in the rsync command, iconv is not active or supported by the Syno.
    Any suggestion to overcome this problem?
    Thanks in advance

  13. Rene
    January 7th, 2013 at 10:45 | #17

    @ Nic
    Hi Nic,
    Thanks a lot for the instructions. But just yesterday I found out that my problem is not with rsync but with the mounting of the windows drive. Already after the mount, the characters were wrong. So for rsync it was like “grabage in, garbage out”. But just 10 minutes ago I found that the problem was that I tested with mount.cifs but used spaces in my command line and therefor the iocharset was not “seen”.
    Now I use simple mount:
    mount //$IPAddress/$SourceShare $MountDir -t cifs -o username=$Username,$Password=,iocharset=utf8
    and that works fine.
    Thanks again.

  14. Romain
    January 13th, 2013 at 07:13 | #18

    Hi Nic,

    Thank’s for this great solution, it’s really working fast. I had some issue at first like BillB but it was because I tried not to use the root account. I ended using it and it works.
    But I can’t use the validation-script, because I have the following error:

    /root/.ssh/validate-rsync.sh: line 1: ?#!/bin/sh: not found
    protocol version mismatch — is your shell clean?

    If i had the line :

    command=”/root/.ssh/validate-rsync.sh /volume1/backup/WIND”

    In the authorized_keys file.
    I can’t figure out why I get this error but It is certainly because of my config and not because of your script.
    I checked the /root/.profile file and it seems ok (with /opt/bin and /opt/sbin in it).

    If you have any clue it would be nice, for now I am using your script without the restriction to rsync.

  15. January 16th, 2013 at 20:03 | #19

    Are you sure validate-rsync.sh starts with “#!/bin/sh” (without quotes) exactly?

    Also, make sure the line-endings are in Unix mode and that the file is in ANSI (not UTF-8) mode. Both of those have caused me problems before.

  16. Jan-Paul
    February 17th, 2014 at 07:23 | #20

    Tried and tested with success, thanks for sharing. My only question would be if you have any advice on how to escape spaces in the path names as I am trying to backup the “My Documents” folder. ‘%rsyncConnectionString%:%rsyncPath%/My Documents/’ doesn’t work as I would have expected. Any tips on this issue?

    Kind regards, Jan-Paul

  17. February 19th, 2014 at 13:45 | #21

    @ Jan-Paul
    Two things to try Jan-Paul.

    One, you can try to escape the space with the \ character. eg: My\ Documents

    I’m not sure, but DOS globs (wildcards) might work too, so you could potentially swap the space with a question mark (?‘) eg: My?Documents

    Hopefully one of those will work for you!

  18. LysanderM
    January 22nd, 2016 at 05:48 | #22

    Nic :Hi Rene. I haven’t had this problem yet (or haven’t noticed it with my files yet), but I was able to install iconv by doing the following: 1) Install ipkg (Possible instructions here http://forum.synology.com/wiki/index.php/Overview_on_modifying_the_Synology_Server,_bootstrap,_ipkg_etc#How_to_install_ipkg) 2) Make sure you have gcc and make installed: ipkg install gcc make 3) Download iconv-1.14.tar.gz: http://www.gnu.org/software/libiconv/#TOCdownloading 4) Extract: gunzip iconv-1.14.tar.gz && tar xfv iconv-1.14.tar 5) Configure and make: cd iconv-1.14 ./configure make && make install

    This is brilliant, thanks Nic.

  19. Justin
    February 5th, 2016 at 15:35 | #23

    Not Sure what I am missing.

    When I run the the script I get two errors. once is Could not create directory ‘/home/username/.ssh’

    It then says it cannot establish the authenticity of the host and comes up with a ECDSA key fingerprint and asks if I would like to continue. When Yes is entered it tells me that it could not add the host to the list of know host and then proceeds to upload files.

    I am guessing I am missing something simple but for the life of me I cannot find it.

    Help would be awesome.

    Cheers,

    Justin

  20. February 7th, 2016 at 09:19 | #24

    @ Justin
    Hi Justin. I think it might be a Cygwin “home” issue. Try the solution here? https://superuser.com/questions/735801/ssh-could-not-create-directory-home-username-ssh

  1. No trackbacks yet.