Gleicher Enterprises

This page describes some of the ways in which HSI has been used to perform non-obvious functions. If you have a request for a topic that is not covered in this section, or you have a nifty trick to share, please send email to the HSI team so that we can post it here.

• How to pipe an HPSS file into "cmp", "head", "less", etc.

It is often desirable to retrieve a file from HPSS immediately after storing it, in order to compare the HPSS file with the original local file, or to view the contents of an HPSS file using a tool such as "less".  The most obvious way to do this is to retrieve the HPSS file to a local file, and then use the standard operating system utilities to operate on the local file.

However, HSI provides the ability to pipe the output of a "get" command into a pipeline of one or more operating system utilities without having to write the file back to the local filesystem.  This can help to avoid problems with disk quotas, insufficient local filesystem space, non-writable filesystems, etc. and can also be significantly faster, depending on the filesystem type, than copying the file to a local filesystem and then operating on it.

 

Pipe syntax is described on the "Using PIPES for Input/Output" page.

Here's an example of using a pipe to run the "cmp" utility to compare a local file with an HPSS file:

 

         put my_local_file : my_HPSS_file

         get "|cmp - my_local_file" : my_HPSS_file

 

In this example, "|cmp - my_local_file" is the local file to which the HPSS file called "my_HPSS_file" is to be renamed, using the local : HPSS rename syntax.  Since the local filename contains spaces, it must be enclosed in quotes.  In addition, since the local filename starts with "|", HSI interprets the remainder of the local filename as a pipe command, and forks a shell to receive the output of the command.   

 

In this case, the "cmp" command interprets the first argument, "-", as standard input.  Other utilities that read from standard input may use a different parameter to represent standard input, or may simply read from standard input if no filename arguments are provided.

 

The "-q" command line option, or the "progress" command toggle, can be used to disable file transfer progress messages that can interfere with the output of utilities such as "head".

 

Note:  It is necessary to type "exit" when piping output into some utilities, such as head, in order to exit from the utility back to the HSI command prompt, even if the entire HPSS file is piped into the utility in less than one full screen.

• How to retrieve a set of files in optimal tape/position order

  
  

  By default, the "get" command will retrieve the specified list of files in tape and position order.  Also note that it may be necessary to use the "-A" option if auto-scheduling has been disabled at the site.

  
  

  However, this only applies to a single "get" command - if multiple "get" commands are issued, each command is independent of any others, and only the list of files specified by the single "get" command will be optimized for retrieval. .

  
  

  The easiest way to optimize a single "get" command is to use the "here-document" form of the command, by creating an IN file that looks like this:

  
  

  get << EOF

     file1

     file2

     file3
  ...
  EOF
  
  

• Working with pathnames containing special characters
  
  
  A common problem is working with filenames containing spaces and special characters that are normally used for pattern-matching. An example of this is shown below:
  
  
  

  Run-202 Future Trends in Automobile Design.pdf

  
  

  Pathnames that contain whitespace characters can be enclosed in single or double quotes, or the whitespace characters can be escaped by backslashes.

  For example:

  
  

  ls "Run-202 Future Trends in Automobile Design.pdf"
  

  or
  

  ls Run-202\ Future\ Trends\ in\ Automobile\ Design.pdf