...
“StorWIS isn’t intended as an HPC storage service, and we are not allowing heavy computation on access servers. So if you need to run a job with data from StorWIS, you need to move/copy from StorWIS to your WEXAC home folder.”
Create your folder in /homestorwis/archiveLabs/labs/YOUR_LAB/
i.e. /homestorwis/archive/labsLabs/bioservices/bareket/Use this command to archive a directory into a tar file, and to save it in your storwis folder.
Example:
tar czf - DIRECTORY_NAME | split --bytes=100G - /homestorwis/archive/labsLabs/YOUR_LAB/DIRECTORY_NAME.tar.gz.
...
To list the contents of the tar files, run:
cat /homestorwis/archiveLabs/labs/YOUR_LAB/DIRECTORY_NAME.tar.gz.* | tar tzf -
To extract the folder to the current directory, run:
cat /homestorwis/archive/labsLabs/YOUR_LAB/DIRECTORY_NAME.tar.gz.* | tar xzvf -
Important notes:
archiving must be done from an “access” server
Some folders should be archived without “zip” (tar cf instead of tar czf) because they already contain zipped files
archiving does not always work well with the bsub command
Some file permissions can be problematic for specific files which were created by the sequencing server, or UTAP
More info in the Wexac page: https://hpcwiki.weizmann.ac.il/en/home/general/backup_restore