LFTP - sophisticated file transfer program



LFTP is a file transfer program supporting a number of network protocols : It is free software, distributed under the GNU GPL license (details).

The mirror functionality (source) :

lftp "ftpLogin:ftpPassword@ftpServer" -e "mirror --verbose=2 --continue --delete --log=/path/to/logFile --parallel=10 --allow-chown --only-newer --exclude=excludedContent /source/directory/on/ftp/server /destination/directory/on/local/server; quit"

More about --exclude.

Keep in mind that FTP admins frequently chroot jail users when specifying the source directory as well as the excluded contents : all paths are relative to the user's home directory !

LFTP and symlinks (source) :

Very basically speaking (this is QnD, needs polish), symlinks are supported, BUT absolute paths found in symlinks may be broken once a chroot has been applied. Keep in mind that a user's home directory becomes / afterwards and symlinks must handle this.