HSI Man Page

NAME

HSI − Hierarchical Storage Interface −


SYNOPSIS

hsi [options] [command [;command [;...]]]


Options are:


-help | -? .... Displays a built-in synopsis of HSI usage. If used as the only command line option, HSI exits instead of entering interactive mode.


-a acctID | acctName ......Specifies the account ID or account name to be used for creating new files and directories.


-A auth_mechanism ..... Specifies the authentication mechanism to be used when HSI starts up. Only authentication mechanisms that are available on both the client and HSI Gateway Server can be used successfully.

Valid mechanisms are:


   • combo - user name and password (or SecurID token if RSA SecurID tokens are in effect) will be used

   • kerberos - specifies that MIT kerberos will be used for authentication

   • keytab - specifies that a Unix or Kerberos-style keytab will be used for authentication

   • gsi - specifies that the Globus GSI certificate protocol will be used.

   • local -(AIX only) - this protocol is used for sites that allow users to authenticate using their AIX (or LDAP) passwords on the HSI Gateway Server machines.


-c cred_file ........ Specifies the credentials cache file to be used if kerberos authentication is used.


-d debug_level ....... Specifies debugging level to enable when HSI starts up. debug_level is an integer value between 0 and 4. Level 0 (the default) specifies no debugging, levels 1-4 specify successively higher debug levels.


In general, HSI debugging output is useful only to the developers for troubleshooting problems


-e ...... "Echo" flag. If enabled on the command line, or by the "echo" command, HSI will display command lines that are read from IN files.


-E editor-style ....... Specifies the editing style to be used when editing command lines. editing-style is either vi or emacs. The default, if not specified in the .editrc file, is "-E vi".


Note: this option is effective only if HSI was built with line-editing enabled.


-G globus_grid_proxy_path ........If Globus GSI authentication is used for authentication, (for example, if "-A gsi" is specified), this option can be used to specify an alternative path to the proxy created by the grid-proxy-init command.


-h hostname ...... Directs HSI to attempt to connect to hostname, which can be either a name or an IP address. 


-k keytab_path ...... If keytab authentication is used for authentication, the path to the keytab file can be specified by this option. [Note: the keytab file will not be used by HSI if it has other than <owner> permissions.]


-l login_name ...... This option can be used to specify the user’s login name. The normal default is to use the local login name for the client host on which HSI is being run. This option overrides any login name specified in the hsirc file(s). If no login name is specified, the default is to use the local login name on the client machine on which HSI is being run.


-O pathname ........ This option is intended for use by applications that run HSI via the "system" or "popen" calls. It causes HSI to write all output that would normally be written to the TTY or stderr to the file specified by pathname. The option takes effect immediately when it is encountered, and normally is the first option on the command line, if used.


-o ..... overrides the <no-login> flag, if set by the HPSS administrator. This option can only be used by root. It is deprecated, and will be removed in a future release


-P ......... popen command mode. This option is similar to the -O option, but causes all output to be directed to stdout, which is normally redirected to a process that starts HSI with the popen(3) system call. It also results in setting "quiet" (no extraneous messages) mode, disabling verbose response message, and disabling interactive file transfer messages.


-p port ....... Specifies the port number to be used when connecting to the HSI Gateway Server. This option is only effective if the "-h hostname" option is specified.


-q ........ Enables "quiet" mode. In this mode, extraneous messages such as the interactive file transfer progress messages, as well as the startup "Message of the Day" are not displayed


-s site_name ...... Specifies the global or private hsirc stanza name to be used. site_name must match the name of one of the stanzas in either the global hsirc file, or the user’s private $HOME/.hsirc file


-V .......... Prints HSI version information without requiring a connection to the HSI Gateway Server. If used as the only command line option, HSI exits instead of entering interactive mode


-v .......... Enables verbose mode for command output. Verbose mode is used for providing interactive feedback, such as displaying markers on the listable output file when performing operations such as puts or gets, or searching for files via the find and du commands.


DESCRIPTION


HSI is a utility that communicates with the High Performance Storage System (HPSS), providing a friendly interface to make it easy to transfer files and manipulate files and directories using familiar UN*X-style commands.


HSI supports recursion for most commands (-R parameter), as well as CSH-style support for wildcard patterns, interactive command line and history mechanisms, among its many other features.


FTP COMPATIBILITY/DIFFERENCES


HSI supports several of the commonly used FTP commands, including "dir","get","ls","mdelete","mget","put","mput" and "prompt", with the following differences:


    • The "dir" command is an alias for "ls" in HSI. The "ls" command supports an extensive set of options for displaying files, including wildcard pattern-matching, and the ability to recursively list a directory tree


    • The "put" and "get" family of commands support recursion


    • There are "conditional put" and "conditional" get commands (cput, cget)


    • The syntax for renaming local files when storing files to HPSS or retrieving files from HPSS is different than FTP. With HSI, the syntax is always 

     "local_file :
HPSS_file"

and multiple such pairs may be specified on a single command line.


With FTP, the local filename is specified first on a "put" command, and second on a "get" command. 


For example, when using HSI to store the local file "file1" as HPSS file "hpss_file1", then retrieve it back to the local filesystem as "file1.bak", the following commands could be used:

    put file1 : hpss_file1
    get file1.bak : hpss_file1

With FTP, the following commands could be used:

    put file1 hpss_file1 

    get hpss_file1 file1.bak

    • The "m" prefix is not needed for HSI commands; all commands that work with files accept multiple files on the command line. The "m" series of commands are intended to provide a measure of compatibility for FTP users.


STARTING HSI


You can use HSI interactively by entering the command "hsi" with no arguments. Then enter HSI requests in response to the question-mark (?) prompt (or whatever the prompt string has been set to). Terminate the HSI session with the end command, or one of its several aliases, such as "bye" or "quit".


You can also use HSI in single execute line mode (also known as "one-liner" mode) by including the HSI request(s) on the command line, following the program name hsi. Multiple commands can be specified, separated by the semicolon ";" character. The command line may need to be enclosed in single or double quotes to protect it from expansion by the shell program that launches HSI.


Single-line execution is often used when HSI is run from within Unix scripts and pipelines. An example of using HSI with two commands on the execute line might be:

    hsi "cd /users/project1;ls -l *.c"

See OPERATING SYSTEM CONSIDERATIONS below for more information on using hsi in scripts.


HSI HELP


Within an HSI session, help for a command can be obtained by either

    • Entering a command with no parameters, e.g. "put"

    • For commands that do not require parameters, enter the command name followed by "-?", for example:

     "ls -?"


There is also a "help" command that will display a short summary of available HSI commands.


HSI COMMAND SUMMARY

Aliases for commands, if any, are shown in parentheses following the "real" hsi command.


Commands may be abbreviated - the minimum abbreviation for each command is shown by the characters preceding the "[" (if any) for the command name.


add

See "mkdir"


ado[pt] [-l] [-h] [keyset]


Adopt a set of keyword values. If no keyset is specified, adopt the system default keyset.

-l adopt local keyset.
-h adopt HPSS-resident keyset.


anno[tate] [-R] [-e] [-d] [-f] [-A "string"] path ...

Adds annotation text to existing files or directories

-R recursively traverse directories in the specified path(s)

-e erase any existing annotation

-d operate only on directory objects

-f operate only on file objects

-A annotation string (including metacharacters). An empty string is treated as if the -e option was specified


The annotation string may be a maximum of 255 characters. It is silently truncated during expansion if it exceeds this length. 
The annotation string may contain backslash-escaped characters which are replaced with their normal ASCII equivalents as follows: 

  • \\a alert -> bell character 
  • \\n newline 
  • \\f formfeed 
  • \\r carriage return 
  • \t horizontal tab 
  • \\v vertical tab 
  • \\ backslash 
  • \\? question mark 
  • \\' single quote 
  • \\" double quote 

Note that 2 consecutive backslash characters are required in order to enter one of the above, since the HSI parser treats the first backslash as a "quote" character while initially scanning the command. The second backslash and the character following are then evaluated by the code that the annotate command. 

bell

Toggle sounding a bell when keyboard input is needed.


bye

See "end"


c[opy]

See "cp"


cd[0..9] [drive:]path : change a working directory path

cd -? : print usage message

Changes current HPSS Working Directory to dirname. The default is your HPSS home directory. Working directory 0 is used if the directory number is not specified. Working directory number. 0 is used if the directory number is not specified.


"cd" with no parameters is semantically equal to "cd0 ~". The following are exactly equivalent: 

    cd0 or cd - or cd ~


cdl[s] [options] [path ...]

Change the current directory to pathname and list its contents, using the listing options specified by options. See the "ls" command for listing options.


cget [options] [localfile : ]hpsspath ...

Conditionally copy an HPSS file hpsspath to localfile if it does not exist locally. Default localfile is hpsspath.

See "get" for a description of all of the command options.


chac[ct] [-R] new-acct path ...

Change account ID for HPSS file path to new-acct.

Options are:
-R recursion flag


chcos [-h] [-N] [-R] new-cos | auto path ...

Change Class of Service for path to new-cos

Options are:

h : Follow symbolic links during recursion. The default behavior is not to follow symbolically-linked directories.
-N : disables auto-scheduling of files (by Volume and Position). The normal default is to organize the list of files by Volume ID and position, in order to optimize tape mounts
-R : recursively traverse directories in the specified path(s)
-Z : internal autoscheduler debug flag. Displays scheduling results but doesn’t issue the HPSS chcos call(s)


new_cos : specifies the COS ID to be used, which must be one which you are eligible to use for the HPSS connection(s) specified by the "path" parameter(s) (use "lscos" to see a list of COS IDs).


COS ID 0 ("chcos 0") is always permitted; it removes files from the HPSS chcos queue.

Use "chcos auto" to use auto-COS selection to choose the new COS ID, based upon current settings for # copies, account ID, group ID, and user ID


chgrp [-h] [-R] group-name path ...

Change group owner for path to group-name.

Options are:

-h  For symlinks, change ownership of symlink instead of the object which it references. [Note: HPSS does not provide this capability, so symlinks are silently ignored if this option is specified]

-R recursion flag


chmod [-d] [-f] [-h] [-H] [-R] mode path ...

Change permissions for HPSS files.

-d changes apply only to directory objects
-f changes apply only to file objects
-h suppresses a mode change for a file or directory pointed to by a symbolic link
-H : If the -R option is specified, symbolic links on the command line are followed. Default is not to follow symlinks for any path specified on the command line. Symbolic links encountered during recursion are never followed.
-R descends directories recursively, changing the file mode bits of each directory and of all files matching the specified pattern, taking into account the other flags. If a symbolic link is encountered, and the link points to a directory, the file mode bits of that directory are changed, but the directory is not further traversed.


The mode of each named file is changed according to mode, which may be absolute or symbolic. An absolute mode is an octal number constructed from the logical OR of the following modes:

    0400 read by owner
    0200 write by owner
    0100 execute (search in directory) by owner
    0070 read, write, execute (search) by group
    0007 read, write, execute (search) by others


A symbolic mode has the form:

    [who] op permission [op permission] ...

The who part is a combination of the letters u (for user’s permissions), g (group) and o (other). The letter a stands for all, or ugo. If who is omitted, the default is a but the setting of the file creation mask (see umask(2)) is taken into account.


op can be + to add permission to the file’s mode, - to take away permission and = to assign permission absolutely (all other bits will be reset).


permission is any combination of the letters r (read), w (write), x (execute), X (set execute only if file is a directory or some other execute bit is set). Letters u, g, or o indicate that permission is to be taken from the current mode. Unlike Un*x systems, a value must be specified with =; to remove all permissions, specify either absolute or -rwx.


When the -R option is given, chmod recursively descends its directory arguments, setting the mode for each file as described above. When symbolic links are encountered, their mode is not changed and they are not traversed.


chown [-h] [-R] owner-name path ...

Changes owner of path to owner-name.

-h suppresses a change for a file or directory pointed to by a symbolic link. Note: this option, if specified, currently causes symbolic links to be ignored with a warning message, as HPSS does not provide the capability to change the ownership of a symbolic link
-R specifies recursion


close [connectionID|drive: ...]

Close a connection to an HPSS server

connection ID : numeric connection id
drive: logical drive prefix, including colon ":" character

Use lsc[onnections] for list of current open connections. If no connections are specified, the current connection is closed. It is an error to attempt to close the last active connection.


connect

See "open"


cp | c[opy] [options] sourcefile sinkfile
cp | c[opy] [options] source ... sinkDirectory

Copies one file to another, or copies files into a directory


Options are:
-B : create backup of target file by renaming if it exists (same as "backup=on") [**Note: not yet implemented]
-C : cache purge option. If specified, purge source files from HPSS disk cache after a successful copy. Normally used only when it’s expected that files will only be read once (or infrequently), to help optimize HPSS disk cache use.
-f : force removal of the target file instead of renaming (same as "backup=off")
-h : copy symbolic links (default is to read through symlinks)
-i : prompt before copying files (interactive mode)
-m : interhpss copy method ("local" or "server"). Default is "server"
-n : only copy files modified within the specified number of days
-p : preserve timestamp from source file
-R or -r : recursively copy SourceDirectories
-S : disable staging of the source file (attempts to read directly from tape). This option may be useful for large files that will only be retrieved once (or infrequently) to avoid the time and space that it takes to stage the file(s) onto HPSS disk cache.
-x : use extended I/O calls. The "which" parameter is "source", "sink",or "both". This option is normally only used when "firewall" mode is in effect, and designates whether firewall-type I/O should be used when reading ("source") files, writing files ("sink") or both ("both"). One scenario where this is useful is copying files between HPSS systems, when one or the other HPSS systems lives behind a firewall.


cput [options] [localfile : ] hpsspath ...

Conditionally save or replace HPSS file hpsspath with localfile if it has been updated.

See the put command description for a description of options


cren[ame] [-d] [-ocos id[,id ...]] [-ncos id[,id ...]] [-s] new_path orig_path

Conditionally rename and optionally delete files

-d : delete original file if the rename operation succeeds
-s : suffix string to append to orig_path when renaming it. Default is "~"
-ocos : Filter option. If specified, the Class of Service of file "orig_path" must be one of the comma-separated COS IDs specified, or the command will fail.
-ncos : Filter option. If specified, the Class of Service of file "new_path" must be one of the comma-separated COS IDs specified, or the command will fail.


debug level|on|off

Set debugging level

level : a number in the range 0 to 5. 0 disables debug messages, levels 1-5 enable increasingly higher levels of debug, roughly corresponding to the following:
1 ..... event and high-level information messages
2 ..... normal debug messages that help track the flow of command processing
3 ..... higher level of debug used for inner loops, table traversals, etc.
4 ..... I/O debug - mover message traces
5 ..... trace-level debug
on .... equivalent to "debug 1"
off ... equivalent to "debug 0"


delete

See "rm"


dir

See "ls"


du [ -p ] [ -s ] [ -a ] [-b date] [-e date] [-s] [-w n] [path...]

Summarize HPSS file space usage in specified path or current directory

du gives the number of kilobytes contained in all files and, recursively, directories within each specified directory or file name. If name is missing, ‘.’ is used.


-a causes an entry to be generated for each file. If neither -s or -a is specified, an entry is generated for each directory only.
-b date only counts files written since the specified date. date is of the form yyyy/mm/dd.
-e date only counts files written before or on the specified date. date is of the form yyyy/mm/dd. Note that the -w option, if specified, must reference a date prior to or equal to the one specified by the -e option or an error will occur.
-k print values in kilobytes (1k=1024). Default is 512 byte blocks
-s causes only the grand total to be printed.
-w n only counts files written within the last ’n’ days


dump [-R] [-x] path ...

List metadeta information for HPSS Nameserver objects.

Options:
-R : recursively traverse directories in the specified path(s)
-x : dump extended attributes for object

echo

toggle for displaying command input from IN files

end

(ex[it], q[uit],bye) Terminate HSI execution


erase

See "rm"


exit

See "end"


find pathname-list expression

Find nodes in the directory structure matching specified criteria. (The implementation is a subset of the Un*x version of find(1), but the syntax is almost identical.


FIND recursively descends the directory hierarchy for each pathname in the pathname-list, seeking files that match a boolean (logical) expression written in the primaries given below. In the descriptions, the argument n is used as a decimal integer where +n means more than n, -n means less than n, and n means exactly n.


-name filename True if the filename argument matches the current file name. Wildcard characters are allowed, with or surrounding quotes ("" or ’’).


-perm onum True if the file permission flags exactly match the octal number onum. Only the lowest 3 octal digits are significant.


-cos cosid True only for plain files whose Class of Service ID matches that specified by cosid


-prune Always yields true. Has the side effect of pruning the search tree at the file. That is, if the current path name is a directory, find will not descend into that directory.


-type c True if the type of the file is c, where c is one of:

    d for directory,
    f for plain file,
    l for symbolic link


-user uname True if the file belongs to the user uname. If uname is numeric, it is taken as a user ID.


-nouser True if the file belongs to a user not in the passwd database.


-group gname True if the file belongs to group gname. If gname is numeric, it is taken as a group ID.


-nogroup True if the file belongs to a group not in the group database.


-size [+ | -] n
If - is specified, true if the filesize is < n bytes.

If + is specified, true if the filesize > n bytes.
Otherwise, true if the filesize is exactly n bytes.


-atime n True if the file has been accessed in n days. Note that the access time of directories in pathname-list is changed by the find command.


-mtime nfR True if the file has been modified in n days.


-ctime n True if the file has been changed in n days. ‘‘Changed’’ means that the file’s "st_mtime" field has changed.


-print Always true; the current pathname is printed.


-ls Always true; causes current pathname to be printed together with its associated statistics.


-newer filename True if the current file has been modified more recently than the HPSS file filename.


-depth Always true; causes descent of the directory hierarchy to be done so that all entries in a directory are acted on before the directory itself.


-exec command (DEFERRED - intended for use ’get/put’, etc) True if the executed command returns a zero value as exit status. The end of command must be punctuated by an escaped semicolon. A command argument {} is replaced by the current pathname.


-ok command (DEFERRED) Like -exec except that the generated command is written on the standard output, then the standard input is read and the command executed only upon response y.


fire[wall] [-on | -off] [-c] [-s] [drive ...]

Enable or disable firewall mode for a connection. When firewall mode is enabled, store-and-forward I/O is used for file transfers instead of the normal mode where HPSS movers connect to HSI. In firewall mode, auto-scheduling is disabled for the "get" command family.


fr[ee] [-h] [-l] keyset

Delete the kept set of keyword values, keyset.

-l deletes local keysets.

-h (default) deletes HPSS-resident keysets.


g[et] [options] [localfile :]hpsspath ... [<<MARKER]

(mget, rec[v]) Copy a HPSS file hpsspath to localfile. Default localfile is hpsspath. NOTE: mget prompts for "yes/no" in interactive mode. Use prompt to toggle interactive mode.


Options are

-A : enable auto-scheduling for retrievals in order to optimize tape mounts
-B | -b : backup option. Renames any existing local file by appending "~"
-C : purge source file from disk cache after file(s) are copied successfully
-F on | off : enable or disable the use of HPSS Local File Mover I/O, overriding any configuration option(s)
-h : symlink option. If specified, create local symlink if HPSS symlink. If not set, reads through HPSS symlinks when copying files to local filesystem
-k : keep partially transferred file(s) if error(s) are encountered
-L path : specifies pathname to a file containing a list of file segments to be retrieved.
-N : disable auto-scheduling of retrievals
-O tuple : partial file transfer specification. See the usage notes below for the format of the "tuple" parameter. Multiple -O options can be specified for a command.
-p : preserve timestamp. Attempts to copy the HPSS file’s timestamp to the local file.
-Q : if running as local root user, attempts to preserve the HPSS owner and group on the local file(s).
-R : recursively copy directories
-S : disable staging of the HPSS file(s); read directly from tape if file is not already in HPSS disk cache.
-T on | off : enable or disable the use of the HPSS Transfer Agent for I/O
-t : retransmit request. Appends to the local file, using the local file's size as the starting offset within the HPSS file.
-U | -u : update option. Only copy HPSS file to the local file if the HPSS file timestamp is newer.sp 1

MARKER : provides sh-style "here"-document syntax for specifying filenames. Lines following the MARKER, which must be the last token on the line, contain pathname (which may include wildcards). The list is terminated by a line containing MARKER as the first token on the line.

Note: If -O,-L is specified, offsets apply only to the first file transferred.


glob

Toggle enabling/disabling wildcard pattern-matching for filenames. The normal mode is "enabled".

If globbing is disabled, all pathnames that are specified must exactly match, and wildcard characters * ? { } [ ]   are treated as normal characters in pathnames.


gro[ups]

List groups associated with the current login context.


help

Displays brief "help" information.


hist[ory] [-e ename] [-nlr] [first [last]] Display or edit command history

or

hist[ory] -s [old=new] [cmd_string]

-e: specifies editor name to use with first form of the command. If not specified, then the environment variables "HISTEDIT" or "FCEDIT" are checked, and if neither is set, then "/bin/edit" is used.

-l: specifies <listing> action to display the commands on the listing file

-n: if specified, suppresses command numbers when editing

Aliases: r - alias for "hist -s"


id [-u | -G | -g [-n]] [user

Display current login identity information

-G : if specified, causes group list for current ID to be displayed
-g : if specified, causes group ID to be displayed
-u : if specified, causes user ID to be displayed
-n : if specified, causes -g or -u option to display names instead of numbers

If [user] is specified, causes info to be displayed for the specified user. If not specified, info for the current login or SU identity is displayed.


idle[time] [value]

Set interactive session timeout value

value: optional idle timeout value, in seconds. If value is zero or negative, e.g. 

     timeout -1

then an infinite timeout is used. If value is not specified, the current idle timeout setting is displayed.
The default value for this setting is 1800 seconds (1/2 hour).


in localfilepath

Read hsi commands from local file localfilepath.

The IN command is not allowed from within an IN file.


k[eep] [-h] [-l] keyset

Save a set of keyword values as keyset.

-l saves local keysets.
-h (default) saves HPSS-resident keysets.


lcd localpath

Change local working directory to localpath.

Note: wildcard characters in pathname are currently not expanded.


lcdl[s] [options] [path]

Change local working directory to path and list the contents.

options: Local list options. These may differ, depending upon the local host’s version of UNIX.
Note: wildcard characters in pathname are currently not expanded.


lls [options] [localpath]

List local (worker) directory localpath. Default is local current working directory.

options: Local list options. These may differ, depending upon the local host’s version of UNIX.
Note: wildcard characters in localpath are currently not expanded.


lmkd[ir] [-p] subdirname ...

(lmd) Make a new subdirectory node called subdirname on local filesystem.

-p makes any missing intermediate subdirectories in the subdirname path


list

See "ls"


lpwd

List local current working directory.


ln [-f ] [- h] [-n ] [ -s ] sourcename] [targetname]
ln [-f ] [- h] [-n ] [ -s ] sourcename1 [ sourcename2 ... ] targetdirectory

Create symbolic link for HPSS file sourcename.

Options:
-f : Unlink any already existing file, permitting the link to occur
-h : If the target_file or target_dir is a symbolic link, do not follow it. This is most useful with the -f option, to replace a symlink which may point to a directory.
-n : Same as -h, for compatibility with other ln implementations.
-s : Create a symbolic link.


log [[>[!]]file] [>>file] [-]

Write HPSS log output to file.


The following command formats are recognized:

 

log : write HSI loggable output to a file

log -  ...... .   close logfile if open, revert back to <no logfile> mode

log file ..... write HSI loggable output to <file>

log > file ... same as "log file"

log >! file .. same as "log file"

log >> file .. append HSI response output to <file>


l[s] [options] [path]

(list) Display information about a directory or file path.

options are the following:


-a list all entries, including "hidden" files
-c use time of last modification for sorting (deferred)
-d if file is a dir, list its name instead of its contents
-l (letter "l") long list format
-p put a slash after each name if the file is a directory
-r reverse alpha or age sort order, as appropriate (deferred)
-s display size as well as name if -1 (numeral 1) option used
-u use time of last access for sorting instead of last modification (deferred)
-x multicolumn output format, with entries sorted across page
-A display annotation string
-C multicolumn output format, with entries sorted down the columns
-F puts a / after directory filenames, and an * after the name if it is executable (deferred)
-H print headings on long listings

-L n For -P listings, specifies the optional hierarchy level (0-4) to be displayed.

Default is the first tape level of the hierarchy.
-O print unordered "-l" or "-1" format listings
-P : print one line per node with volume/position/VSN list/COS/file family/subsystem info
-R recursively list the directories
-T w|r|c|m select time field to display for long ("-l") listings
w: last write (default) r: last read c: create m: last metadata updat
-U print HPSS-specific information
-V print volume info for 1st tape leve
-X print extended volume info (for all levels)
-1 (numeral "one")) forces one name-per-line list


lscon[nections]

Display active HPSS server connections


lscos [-n] [-N namedCosList] [drive ...]

Lists available classes of service or Named COS information for specified connections

Options: -n : shows Named COS Lists available for the connection
-N : list COS IDs associated with the specified Named COS list
drive : one or more logical drive IDs, e.g. "lscos A: C:"


lsfset

See "lsfileset"


lsfiles[ets] [-l] [drive ...][filesetname]

Display Core Server filesets

-l long listing
drive logical drive for the connection. Default is the current connection


lsjun[ctions] [-s subsysID] [drive ...]

Display Core Server junctions

-s Subsystem ID (default = 1)
drive: logical drive for the connection. Default is the current connection.


lssi[tes] [-a] [-c] [-h] [-m] [-p] [-s site[,site ...]] [drive ...]

Display site entries from hsirc file(s)

-a list all info for sites
-c list server host name and connection info
-h list just server host info
-m list authentication method(s) defined for this site
-p list principal info for the site
-s list info for the specified site(s). The parameter may contain either a single site name, or a comma-separated list of sitenames. Sitenames are case sensitive.
drive optional space-separated list of drive letter(s) for which the site info is to be displayed, e.g. "a: z:"


mdelete

See "rm"


mget

See "get"


md

See "mkdir"


mdel

See "delete"


mget

See "get"


mig[rate] [-h] [- L level | -l level] [-F | -f] [-P | -p] [-R] [-S] path ...

migrate files from disk cache to tape

-h : if specified, treats symlinks as normal files. Default is to skip symlinks
-L or -l : specifies hierarchy level from which to migrate data. The valid range is 0 (default) to 4
-F or -f : force migration even if not needed
-R : [standard option]recursively traverse directories in the specified path(s)
-P : purge file from disk cache after migration
-S : allow staging of file if no disk data.


mkd[ir] [-A "annotation"] [-m mode] [-p] path

(a[dd], md, mkd[ir]) 

Make a new subdirectory node called path.

-A : annotation string to set on newly created or preexisting directories. If -p is specified, and intermediate subdirectories are created, this option only applies to the final subdirectory in the path(s)
-m : mode to use for creation of the directory(s). This must be an octal value in the range 0-7777. The mode value is silently truncated if it exceeds this range.
-p : Creates missing intermediate path name directories. If the -p flag is not specified, the parent directory of each newly-created directory must already exist.


move

See "mv"


mput

See "put"


mv [-f] [-i] [-v] path1 path2

(move, ren[ame]) Move HPSS object path1 to new HPSS object path2.

mv [-f] [-i] [-v] path1 path2 ... dirpath

Move HPSS objects into a directory

-f : force option. Removes existing target if it already exists. Default is not to remove existing target(s)
-i : in interactive mode, pause for confirmation before removing existing target


newa[cct] [accountID | accountName]]

set new account ID for current session or list available accounts if accountID (or accountName) is not specified.


open [options]

Open a new HPSS server connection Aliases: connect

-A auth_method
-c credentials cache filename (kerberos)
-d driveletter: (note ":" is optional)
-h remote host name or IP address
-k password-file (or keytab-file)
-l principal
-p portnumber
-q "quiet" mode (no Message of The Day) during startup
-s sitename

(A list of sitenames defined in the the hsirc or ~/.hsirc file can be obtained by using the "lssite") command.


out [ - | [> | >|] | >>] localfile]

Writes HSI listable output to localfile

out - closes any current output file and reverts to terminal ouptut
out file, out > file, and out >| file all attempt to open and write to a local file called file, overwriting any previous contents
out >> file attempts to append to an existing file


plo[ck] [-h] [-R] [-s] path ...

Lock files in HPSS disk cache

-h : read through symlinks as if they were regular files/directories [default: ignore symlinks]
-R : recursively traverse directories in the specified path(s)
-s : prestage files if necessary prior to locking This command should be used judiciously, as it may cause the disk cache to fill up with files that are not purgeable. Some sites may choose to disable this command - check with your site administrator to determine the local site policy for the use of this command


progress

Toggle display of file transfer progress.


prompt

Toggle for enabling or disabling interactive prompting for multiple file transfers (mget, mput, mdelete commands).


punl[ock] [-h] [-R] path ...

Unlock files the were previously locked in disk cache

h : read through symlinks as if they were regular files/directories [default: ignore symlinks]
-R : recursively traverse directories in the specified path(s)


purge [-F] [-R] [-h] [-l hlevel] path ...

Purge files from HPSS disk cache at top level of a hierarchy

-F : force purge to occur
-h : if specified, treats symlinks as normal files. Default is to skip symlinks
-l : purge data from hierarchy level hlevel (0-4) Default is level 0.
-R : [standard option]recursively traverse directories in the specified path(s)


put [-A "string"] [-B | -b] [-d] [-F on|off] [-h]

[-M mode] [-n] [-P | -p] [-Q] [-R | -r] [-T on|off] [-t] [-U | -u] path ...

Store a file into HPSS, overwriting any existing file with the same name


-A : annotation string for HPSS file (type "annotate ?"for more details)
-B | -b : backup option. Renames existing HPSS file by appending "~"
-d : remove local files after success transfer to HPSS
-F on|off : enable or disable use of the HPSS Local File Mover, overriding any configuration settings in the HSIRC file(s)
-h : symlink option. If specified, create symlink in HPSS if local symlink. If not set, reads through local symlinks when copying files to HPSS
-M mode : specifies octal mode to use for file creations
-n days : only put files modified within last "days" number of days
-P : create intermediate HPSS subdirectories for the file(s) if they do not exist
-p : preserve timestamp. Attempts to copy local file’s timestamp to HPSS file
-Q: if running as the HPSS "root" user, preserve the local owner/group for HPSS files and directories that are created
-R | -r : recursively store directories
-T on|off : enable or disable use of the HPSS Transfer Agent, overriding any configuration settings in the HSIRC file(s)
-t : "re-put" operation. Restart a previously failed operation, using the size of the existing HPSS file as the starting offset.
-U | -u : update option. Only copy file to HPSS if local file timestamp is newer


pwd[0-9] [drive: ...]

List current HPSS working directory path.

Working directory number (0 is the default if none is specified).
drive : logical drive ID. Default is the current connection


q[uit]

See "end"


r

Alias for "history -s". See "history"


rd

See "rmdir"


rec[v]

See "get"


rep[lace]

See "put"


remove

See "rmdir"


ren[ame]

See "mv"


r[eplace]

See "store"


rm [-i] [-R] path ...

(del[ete], mdel[ete], erase) Remove filename from HPSS.

-i Interactively confirm each delete.
-R Recursively remove files and subdirectories in the specified paths.

NOTE: mdelete prompts for "yes/no" in interactive mode. Use the prompt command to toggle interactive mode. "-i" causes a prompt to be issued regardless of the "prompt" setting.


rmdir path ...

(rd, rem[ove]) Remove directory path. This command will fail if the directory is not empty


s[ave]

See "cput"


send

See "put"


se[t] [keyword = value ...]

(defa[ult]) 

Set or display keyword values. These can be set for individual commands by either including them on the command line in the form "keyword=value", or for the entire session by using the command set keyword=value.


The set verb is optional; any line which is of the form keyword = value is interpreted as a set command.

You can also save a keyset (by using the keep command) and reload it later (by using the adopt command) in the same or another session.


   ba[ckup]=on | off automatically renames existing file on get/put. Default is on.

col[umns]=numeric value Specifies number of columns per line. This value is used by the ls command when determining the number of entries per line to list. It is changed whenever the screen size changes.


   copies=n number of copies to store. The default is set by the HPSS administrator in the server configuration file.


   cos[id]=auto | cosid class of services to use. Default is "auto", which selects class of service for you. You should only change this if you need to use a different class of service, for example, to go directly to tape. Use the lscos command to see the classes of service defined for the site; be aware that these are highly site-dependent.


   coslist=name "named Class of Service" to use. Default is no named COS. Named Classes of Services are groups of COSs that are defined by the HPSS administrator. Use "lscos -n" to display a list of named COSs for the current active connection.


   dcreate[mode]=octal_value Octal mode to use when creating directories. This value is defined in the global or private hsirc file’s dcreate_mode setting, or to 0777 by default. The current umask value is used in conjunction with the mode setting in order to yield the effective permissions that are set.


   dirn=pathname (n is omitted(meaning dir0) or 0 to 9) (The default pathname is your HPSS home directory) dir0 or dir assigns the current working directory. dirn sets up a shorthand name for the nth working directory as "pathname’".  This can be used in commands by using the "tic" notation, for  ls 3'.


   family[id]=numeric_value Set default file family ID to be used when creating new files. The default is either 0, or the value defined in the global or private hsirc file(s) for your site.


   fcreate[mode]=octal_value Octal mode to use when creating files. This value is defined in the global or private hsirc file’s fcreate_mode setting, or to 0777 by default. The current umask value is used in conjunction with the mode setting in order to yield the effective permissions that are set.


   iob[ufsize]=numeric_value I/O buffer size to use when transferring files. The default is set by the HPSS administrator when HSI is built, normally 8MB. The numeric_value can optionally be followed by any of the following multipliers, with no intervening space: "k", "kb", "m", "mb", "g","gb", "t", "tb", "p", "pb" for kilobytes, megabytes, gigabytes, terabytes, petabytes. The maximum buffersize is normally 32MB, but is defined at build time by the HPSS administrator.


   lines=numeric_value Sets the number of lines per page for the terminal. This value was used for a curses-based help package for a previous version of HSI, but is currently unused.


   promptl[en]=numeric_value Sets the prompt length in characters


   promptd[irlen]=numeric_value Sets the maximum number of characters to use when expanding the %w metacharacter in the command prompt. This is the number of directory components to include from the righthand side of the pathname if the prompt must be truncated to promptlen characters.


   PS1=string Specifies the prompt string to use for initial command prompting. See the "Prompt String" section for details on the options for string. The default setting for this value is "? " or the value of the PS1 setting in the global or private hsirc file.


   PS2=string Specifies the prompt string to use for continuation lines. See the "Prompt String" section for details on the options for string. The default setting for this value is "> " or the value of the PS2 setting in the global or private hsirc file.


   pwid[th]=auto|numeric_value Sets the parallel stripe width to use when transferring files. If pwidth is set to "auto" (the normal case), HSI automatically determines the optimum stripe width to use. 


You can set these keywords for individual commands, for your session, or in saved keysets.


setc[on] conid

Set current active HPSS server connection

condid:numeric handle to current connection, as shown by the lscon command


setdr[ive] connection|drive newdrive

Change the "logical drive" letter for an HPSS connection

connection: connection handle ID (1-n), as shown by the lsconnections command
drive: current drive letter (a-z, A-Z)
newdrive: drive letter to be assigned (a-z, A-Z)


shared_mem

On/off toggle for enabling/disabling the use of shared memory for I/O buffers for file transfers.


sh[ow] [-l] [-h] [keyset ...]

Display names of kept keysets, or if keyset is specified, shows its contents.

-l lists local keysets.
-h (default) lists HPSS keysets.


showc[on]

See "lsconnections"


st[ore] 

See "put"


sta[ge] [-A] [-i] [-m max] [-N] [-R] [-w] [path ...] [<<MARKER]


Stages file(s) onto HPSS disk cache wihin an HPSS hierarchy


-A :  enable tape mount autoscheduling optimization (the normal default)
-i :   no interactive notification when file(s) have been staged
-m :  max number of stages to issue in a single batch. Default=no limit
-N :  disable optimization of stage requests by VV and position
-R :  recursively traverse directories in the specified path(s)
-w :  wait for each stage to complete (foreground stage). Default is to stage in the background
MARKER :  provides sh-style "here"-document syntax for specifying filenames. 
Lines following the MARKER, which must be the last token on the line, contain one or
more pathnames, (which may optionally include wildcards). The list is terminated by a 
line containing MARKER as  the first token on the line.


su [- | -l | -m] [user | #uid]

Assume identity of another HPSS user

- : same as "-l"
-l : full login with target user’s environment (deferred implementation)
-m : login with current user’s environment (default)
user : login name of the user whose identity is to be assumed
#uid : numeric user id (the "#" prefix is required, with no intervening whitespace) of the user whose identity is to be assumed. Default is root (uid 0)


sudo [-s] [-u user|#uid] [--] command

Issue command(s) as another HPSS user sudo [K | -k | -l] [-v]

-K: "sure kill" option - invalidate the current sudo session and remove the starting timestamp
-k: invalidate the current sudo session, but do not remove the starting timestamp
-l : (deferred implementation) list allowed and forbidden commands for the current active sudo session
-s : SU to the target user and then run the command. Stays in "SU" mode after the command completes
-u : target user login name or numeric uid (# prefix is required, with no intervening whitespace)
-v : validate and extend the current SUDO session’s timestamp
command: HSI command to be executed.


swi[tch]

See "setcon"


t[ouch] [-c cosID] [-M mode] [-R] hpsspath ...

Touch HPSS file hpsspath. Creates HPSS file hpsspath if it does not exist or updates the time last referenced if it exists.

-c : specifies the Class of Service ID to use when creating new files. The lscos command can be used to view a list of available COS IDs.
-M : specifies the octal mode bits to set when creating a new file, for example: -M 0662 to set read/write for owner and group.
-R : recursively traverse directories in the specified path(s)


umask permission-mask

Sets the permission mask to the octal number specified by permission-mask. (See man umask for more information about umask.)


verbose

Toggle "verbose" mode


vers[ion]

Show HSI version information


whoami

Display current HPSS user identity


PROMPT STRING SETTINGS

The default HSI prompt string ("? " for the initial line, and "> " for continuation lines) may be customized by setting the "PS1" and "PS2" keywords, or by specifying the PS1 and PS2 settings in the global or private hsirc files.

For example, setting the strings as follows:
    PS1 = "%d[%H]%w3->"
    PS2 = "[%H]continue: "
would produce a prompt string similar to this for the initial prompt:
    R:[hcdp01]/users/sdsc/gleicher->
and a prompt similar to this for continuation lines:
    [hcdp01]continue:


Within a prompt string, characters are copied literally, except for substitutable parameters of the form %x, where x is one of the following:

    • C : DCE Cellname (currently recognized, but unused)

    • c : current connection ID

    • d : current logical drive ID in the form A:

    • D : system date in yyyy/mm/dd format

    • h : canonical hostname for current connection

    • H : remote HPSS hostname for current connection

    • I : remote HPSS IP address for current connection

    • L : current login principal

    • s : site name (non-DCE version)

    • N : current command number (1:N)

    • T : system time in hh:mm:ss format

    • W[n] : current HPSS working directory for connection, with optional number of directory paths (from right to left in the current directory path) to include as specified by an optional single digit following %W

    • w[n] : current HPSS working directory, with "..." substitution in long directory paths,

with optional number of directory paths (from left to right) in the current directory path to include if "..." substitution is required.

    • % : % character


PATHNAMES

Valid characters for a path parameter are: a-z A-Z 0-9 $%+-._#. 

Wildcard characters are *?^[] and {}, and may be used in all commands which reference either local or HPSS paths, including the "local : HPSS" form. 


Note that names containing wildcard characters must resolve to a single node for some commands, such as SET DIRn=path, or MV path path ... npath, in which npath must resolve to a single directory node.


Unix-style pathname prefixes "~", "." and ".." are also allowed on all filename/HPSS-pathname references.


FILENAMES

HPSS file names and Unix system filenames are both mixed case, and case IS significant.


Maximum values

Path name (including slashes and wild cards) - 

    1024  characters (1023 + 0-byte terminator)
File/Directory name (except wild-card paths) - 

    256 characters (255 + 0-byte terminator)


OPERATING SYSTEM CONSIDERATIONS

Issues to remember when building UNIX scripts to communicate with HPSS:

  • hsi ls outputs a single entry per line and no heading information when "-1" (digit 1, not to be confused with lowercase letter L) is specified 
  • do not use wildcard paths in scripts unless you protect them from the shell with ’ or "". You’re probably better off using IN-files if you can, since they are read directly by HSI without having to first be passed through the shell. 
  • ALL listable output is written to "standard error" unless an OUT command is in effect. 

  • In single-line execution mode for a file transfer request, using "-" as the local file name specifies standard input (or output).  When this option is used, the maximum-sized file (2^64) is assumed; less than this amount may be sent (by closing the input pipe), but not more. 

        tar cf - file1 file2 | hsi save - : project.tar


    In the example above (and anytime the data is tranferred into HPSS via a Unix pipe), HSI cannot determine the file size ahead of time. In this case, if cos=auto is in effect (the default), the file may have to be recopied within HPSS to move it to the correct storage class. If you know ahead of time which class of service is the correct one for a piped file, you can assist HPSS by requesting the appropriate class of service on the command.


    In the example above, if it is known that BigFile belongs to Class of Service 20 (ie, the HPSS administrator has created a class of service whose ID is 20 to contain the files above), the command line could be rewritten as:

        tar cf - file1 file2| hsi save cos=20 - : project.tar

    to force the file(s) to be written to Class of Service 20. In this case, HPSS will not have to recopy the file in order to place it on the correct storage devices.



    • EXAMPLES

    1. Save a "tar file" of C source programs and header files:
          tar cf - *.[ch] | hsi put - : source.tar


      Note: the ":" operator which separates the local and HPSS pathnames must be surrounded by whitespace (one or more space characters)

    2. Restore the tar file source kept above and extract all files:
          hsi get - : source.tar | tar xf -

    3. Get all files in the subdirectory subdira which begin with the letters "b" or "c" (surrounding the wildcard path in single quotes prevents shells on UNIX systems from processing the wild card pattern):
          hsi get ’subdira/[bc]*’
    4. Save your local files that begin with the letter "c" (let the UN*X shell resolve the wild-card path pattern in terms of your local files by not enclosing it in quotes:
          hsi put c*

    5. Delete all files beginning with "m" and ending with 9101 (note that this is an interactive request, not a one-liner request, so the wildcard path does not need quotes to preserve it):
          hsi <RETURN>
          ? delete m*9101

    6. Interactively delete all files beginning with H and ending with a digit, and ask for verification before deleting each such file.
          hsi <RETURN>
          ? mdel H*[0-9]
    7. Interactively descend into the "Source" directory and move all files which end in ".h" into a sibling directory (ie, a directory at the same level in the tree as "Source") named "Include":
          hsi <RETURN>
          ? cd Source
          ? mv *.h ../Include


    AUTHENTICATION

    The set of allowable authentication mechanisms, and the default authentication method to be used, is a site-configuration option. See your HPSS administrator for information pertaining to your site.


    CUSTOMIZATION

    Each user may have a file called .hsirc in their home directory, in order to specify the HPSS Gateway machine’s hostname or IP address, or to specify other options such as the prompt string, whether to enable or disable automatic backups when storing or retrieving files that already exist, and etc.  The hsirc file is documented on the HSI web site.


    SEE ALSO

    HTAR
    The online version of this manual (http://www.mgleicher.us)


    FILES

    hsi
    $HOME/.editrc
    hsirc (global)
    $HOME/.hsirc (user private)
    HPSS.conf (HPSS administrator)
    sudoers (HPSS.administrator)


    BUGS AND DEFICIENCIES

    • HSI should not blindly interpret embedded "=" signs as "keyword=value", as this makes it necessary to backslash-quote any pathname that contains an equals sign.

    • HSI does not expand wildcards for the local pathname(s) for the LLS and LCDLS commands

    Interactive help should be available for individual commands


    AUTHOR

    Michael Gleicher
    Gleicher Enterprises, LLC
    mgleicher@comcast.net


    SUPPORT

    Sites that have a support contract can contact Gleicher Enterprises via their site representative.


    URLS

    http://www.mgleicher.us

    http://www.hpss-collaboration.org

Mar 3, 2007 6:09 AM

Contact: Michael Gleicher (mgleicher@comcast.net)