Friday, April 2, 2010

Using tar with ssh effectively

We know that tar could be used to take backup. We may like to transfer backup to some remote box. Also some time,it is needed to transfer file with directory structure in the remote box.
Lets say we have to transfer many files in the remote box while keeping file permissions and directory structure intact.
We may choose to tar the source directory ,compress it and scp to destination box. We will need to create a v big temp file only to be transferred to the destination box and and removed from source box. Once the file is scp'd to destination box, we will extract the tar'd file to create files in the same directory structure.

Steps :-

(In Src box)

1) Tar the directory in the source file ( tar -cvzf tarfilename.tar.gz /path/of/dir )
2) transfer file to destination box (scp tarfilename.tar.gz user@remote-host)
3) Delete the file ( rm tarfilename.tar.gz)

(In Dest Box)

4) Untar the file (tar -xvzf tarfilename.tar.gz)
5) Remote the file (rm tarfilename.tar.gz)

We can accomplish all these steps in a single command.

Tar has the great ability to send data to stdout/stdin using the "-" (dash) as a filename in the command line. So using that we can string together pipes to send the data to a remote server

tar -cvzf - /dir/to/be/transferred/ | ssh remoteuser@remotehost 'tar -C /path/of/dir -xvzf -'

IN NUTSHELL, WE HAVE REPLACED tarfilename.tar.gz WITH -(DASH)

What this does is pretty simple: it creates a compressed tar file of the /dir/to/be/transferred/ and sends it to stdout (-). We catch stdout with the pipe (|) and then call ssh to connect to a remote server where we execute the remote command tar. Using the remote tar we run a decompress extraction of stdin (-). tar -c changes directory

Great way to transfer a bunch of files securely, as well maintain ownership and permissions.
Its the example of PUSH method.
Following example will show the reverse method.

Lets say we have a backup file in remote host and we want to restore it in local box.
(PULL Method)

ssh user@remote-host "cat /path/to/backup/backupfile.tar.bz2" |tar jpxf -

In this command, we are cat'ing zipped file and output is being redirected to local tar using pipe. So local tar command will replace -(dash) with zipped file and will untar it.

Many other scenarios could be found in http://www.lamolabs.org/blog/1766/pushing-pulling-files-around-using-tar-ssh-scp-rsync/

5 comments:

Shreya said...

hey,it seems u own a nokia 5233.i'm interested in buying that soon.so could i ask 4 sm personal suggestions about it?is the screen robust?can it b used without the plectrum easily?how's the sound quality with a sony headphone?btw,u write good as well!-shreya.

Shreya said...

i hunted u dwn from sulekha.com and fb...

Anonymous said...

When leading goal-setting video game repair, emphasize to children the need for the
delay and he can hunt. Children soon learn that
success is a result of the series' popularity in the form of quiz nights. However, it is not needed. The funny acrid video game repair have seen a trend that was never even thought of. The audience will no doubt form alliances with one lad or the other. Female Saudi athletes allowed to competePerhaps a strange inclusion given the lack of piracy affecting the business model vs.

Also visit my website; 2011 video games

Anonymous said...

thanks! I was trying a bunch of things to push an extract over ssh but its easier to pull it.

sin43k said...

All about tar.gz and tgz on this wiki wikiext.com/tgz