pyslk API reference¶

pyslk v0.4.0

module overview¶

The pyslk package provides Python wrapper functions for the slk and the slk_helpers. The slk is the official command line interface (cli) of the StrongLink system by StrongBox Data Solutions. The slk_helpers was developed at DKRZ and provides some additional functionality beyond the slk.

pyslk.pyslk: basic wrappers for nearly all slk and slk_helpers commands
pyslk.parsers: extended wrappers for a few slk and slk_helpers commands
pyslk.utils: some utility functions
pyslk.exceptions: one exception class used in this package

pyslk.pyslk module¶

pyslk.pyslk provides basic Python wrapper functions for all commands of slk and of slk_helpers. If a command returns an exit code of !=0, the calling wrapper function will throw a PySlkException.

The wrapper functions are made for slk version 3.3.6 and higher. If this package is used with an older slk version, some optional arguments might not work and their usage might cause PySlkExceptions.

pyslk.pyslk.slk_archive(src_path, dst_gns, partsize=None, streams=None, recursive=False, preserve_permissions=False, excluse_hidden=False)¶

Upload files in a directory and optionally tag resources: using directory path and GNS path

Parameters

src_path (str) – search id or gns path of resources to retrieve
dst_gns (str) – destination directory for retrieval
partsize (int) – size of each file to download per stream, Default: 500
streams (int) – number of file part streams to use per node, Default: 4
recursive (bool) – use the -R flag to archive recursively, Default: False
preserve_permissions (bool) – preserve original file permission, Default: False
excluse_hidden (bool) – exclude . (hidden) files, Default: False

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_checksum(resource_path, checksum_type=None)¶

Get a checksum of a resource

Parameters

resource_path – resource (full path)
checksum_type (str) – checksum_type (possible values: None, “sha512”, “adler32”; None => print all)

Returns

stdout of the slk_helpers call

Return type

str

pyslk.pyslk.slk_chmod(gns_path, mode, recursive=False)¶

Change the access mode of a resource or namespace

Parameters

gns_path (str) – namespace or file (full GNS path)
mode (str or int) – new mode/permissions of a file (as known from bash’s chmod)
recursive (bool) – use the -R flag to delete recursively, Default: False

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_delete(gns_path, recursive=False)¶

Soft delete a namespace (optionally all child objects of a non-empty: namespace) or a specific file

Parameters

gns_path (str) – namespace or file (full GNS path)
recursive (bool) – use the -R flag to delete recursively, Default: False

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_exists(gns_path)¶

Check if resource exists

Parameters: gns_path (str) – namespace or resource
Returns: stdout of the slk_helpers call
Return type: str

pyslk.pyslk.slk_group(gns_path, group, recursive=False)¶

DEACTIVATED: Change the group of a resource or namespace

Parameters

gns_path (str) – namespace or file (full GNS path)
group (str or int) – new group of a file (group name or gid)
recursive (bool) – use the -R flag to delete recursively, Default: False

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_hostname()¶

Shows current hostname you are connected to

Returns: stdout of the slk_helpers call
Return type: str

pyslk.pyslk.slk_list(path_or_id, all=False, numeric_ids=False, recursive=False, text=False)¶

List results from search id or GNS path

Parameters

path_or_id (str) – search id or gns path
all (bool) – show ‘.’ files, default: False (don’t show these files)
numeric_ids (bool) – show numeric values for user and group, default: False (show user and group names)
recursive (bool) – use the -R flag to list recursively, default: False
text (bool) – print result to file ‘slk_${USER}_list.txt’, default: False (print to command line / print non-empty return string)

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_metadata(resource_path)¶

Get metadata

Parameters: resource_path – resource (full path)
Returns: stdout of the slk_helpers call
Return type: str

pyslk.pyslk.slk_mkdir(gns_path, recursive=False)¶

Create a directory

Parameters

gns_path (str) – gns path to create
recursive (bool) – use the -R create folders recursively, if the parent folders do not exist

Returns

stdout of the slk_helpers call

Return type

str

pyslk.pyslk.slk_move(src_path, dst_gns)¶

Move namespaces/files from one parent folder to another; renaming is not possible

Parameters

src_path (str) – namespace or file (full GNS path)
dst_path (str) – new parent namespace

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_owner(gns_path, owner, recursive=False)¶

Change the owner of a resource or namespace

Parameters

gns_path (str) – namespace or file (full GNS path)
owner (str or int) – new owner of a file (username or uid)
recursive (bool) – use the -R flag to delete recursively, Default: False

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_recall(path_or_id, dest_dir, recursive=False)¶

Recall files from tape to cache via search id or GNS path

Parameters

path_or_id (str) – search id or gns path of resources to recall
recursive (bool) – use the -R flag to recall recursively, Default: False

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_rename(old_name, new_name)¶

Rename a folder or file; moving is not possible

Parameters

old_name (str) – folder or file name (full GNS path)
new_name (str) – new name (only name; no full GNS path)

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_resourcepath(resource_id)¶

Get path for a resource id

Parameters: resource_id (str or int) – a resource_id
Returns: stdout of the slk_helpers call
Return type: str

pyslk.pyslk.slk_retrieve(path_or_id, dest_dir, partsize=None, streams=None, recursive=False)¶

DEACTIVATED: Retrieve files via search id or GNS path

Parameters

path_or_id (str) – search id or gns path of resources to retrieve
dest_dir (str) – destination directory for retrieval
partsize (int) – size of each file to download per stream, Default: 500
streams (int) – number of file part streams to use per node, Default: 4
recursive (bool) – use the -R flag to retrieve recursively, Default: False

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_search(group=None, user=None, name=None, search_string=None)¶

DEACTIVATED: Creates search and returns search id

Either group, user and/or name can be set or a search_string can be provided.

search_string has to be a valid JSON search string as described in the SLK-CLI manual. Simple double quotes (“) has to be used in the JSON expression (no escaped double quotes, no escaped special characters).

Example for search_string:: search_string=’{“resources.mtime”: {“$gt”: “2021-09-02”}}’

Parameters

group (str or int) – search for files belonging to the provided group name or GID
user (str or int) – search for files belonging to the provided username or UID
name (str) – search files having the provided name
search_string (str) – JSON search query string

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_session()¶

Shows expiration date of your token

Returns: stdout of the slk_helpers call
Return type: str

pyslk.pyslk.slk_size(resource_path)¶

Returns file size in byte

Parameters: resource_path (str) – namespace or resource
Returns: stdout of the slk_helpers call
Return type: str

pyslk.pyslk.slk_tag(path_or_id, metadata, recursive=False)¶

Apply metadata to the namespace and child resources

Parameters

path_or_id (str) – search id or gns path of resources to retrieve
metadata (dict) – dict that holds as keys “[metadata schema].[field]” and as values the metadata values
recursive (bool) – use the -R flag to tag recursively, Default: False

Returns

stdout of the slk call

Return type

str

pyslk.pyslk.slk_version()¶

Lists the version

Returns: stdout of the slk call
Return type: str

pyslk.parsers module¶

pyslk.parsers provides functions that parse the output of the basic wrapper functions of pyslk.pyslk and provide a nicer-to-work-with output.

pyslk.parsers.slk_exists_bool(gns_path)¶

Check if resource exists and return True/False

Parameters: gns_path (str) – namespace or resource
Returns: True if file exists; False otherwise
Return type: bool

pyslk.parsers.slk_list_formatted(path_or_id, all=False, numeric_ids=False, recursive=False, text=False, column_widths=[12, 12, 12, 9, 4, 4, 5, 999], column_names=['permissions', 'owner', 'group', 'filesize', 'day', 'month', 'year', 'filename'], parse_dates=True, parse_sizes=True, full_path=True)¶

Return pandas.DataFrame containing results from search id or GNS path

Calls ‘pyslk.pyslk.slk_list(…)’ and parses the return string into a pandas.DataFrame. All arguments are copied 1:1 into the ‘slk_list’ call except for ‘column_widths’. Assumes six output columns of ‘slk list’ having the widths 12, 12, 12, 9, 4, 4, 5 and 999 (999 => as wide as neceessary).

Note: ‘slk list’ currently only print the modification date and no modification time. The output of this parser might be modified in future when modification times are printed as well.

Parameters

path_or_id (str) – search id or gns path
all (bool) – show ‘.’ files, default: False (don’t show these files)
numeric_ids (bool) – show numeric values for user and group, default: False (show user and group names)
recursive (bool) – use the -R flag to list recursively, default: False
text (bool) – print result to file ‘slk_${USER}_list.txt’, default: False (print to command line / print non-empty return string)
column_widths (list) – fixed widths of columns to be split
column_names (list) – names of the columns in the pandas.DataFrame
parse_dates (bool) – parse day, month and year into a datetime column.
parse_sizes – parse filesize column into bytes integer.
full_path (bool) – add full filepath to filename column.

Returns

output of ‘slk list’ parsed into a pandas.DataFrame with eight columns (permissions, owner, group, size, day, month, year, filename)

Return type

pandas.DataFrame

pyslk.parsers.valid_session()¶

Returns whether session token is valid or not

Returns: True if valid token exists; False otherwise
Return type: bool

pyslk.parsers.valid_token()¶

Returns whether session token is valid or not

Returns: True if valid token exists; False otherwise
Return type: bool

pyslk.utils module¶

pyslk.utils contains utilies needed by pyslk.pyslk.

pyslk.utils.run_slk(slk_call, fun)¶

Runs a provided slk_call via subprocess.run and returns stdout

The argument ‘slk_call’ is used as argument in a call of ‘subprocess.run(…)’. If slk_call is a string then ‘shell=True’ is set in the run(…)-call. Otherwise, the argument ‘shell’ is omitted.

If the command provided via ‘slk_call’ does not exist (e.g. ‘slk’ not available on this system) or if ‘slk’ returns an exit code ‘!= 0’ then a PySlkException is raised.

Parameters

slk_call (str or list of str) – slk call as input for subprocess.run
fun (str) – name of the calling pyslk function; printed when exception is raised

Returns

stdout of the slk call

Return type

str

pyslk.exceptions module¶

pyslk.exceptions provides the class PySlkException, which is the general exception for the pyslk package.

exception pyslk.exceptions.PySlkException¶

Bases: Exception

A PySlkException derived from ‘Exception’

Changelog¶

0.4.0 (2021-11-14)¶

argument updates in pyslk.pyslk (see below)
argument updates in pyslk.parsers (see below)
removed non-existing arguments in pyslk.pyslk.recall
argument name changes (everywhere, see above):
- R => recursive
- n => numeric_ids
- x => exclude_hidden
- a => all
- p => preserve_permissions
pyslk.parsers.slk_list_formatted considerably enhanced
doc updates

0.3.5 (2021-11-10)¶

activated slk retrieve (pyslk.slk_retrieve)
throw proper PySlkException when slk or slk_helpers are not available

0.3.4 (2021-10-25)¶

minor corrections in the documentation
deactivated slk retrieve (pyslk.slk_retrieve)

0.3.2 (2021-10-23)¶

minor editorial corrections and addons in the descriptions and readme

0.3.1 (2021-10-23)¶

updated doc structure
modified package description in __init__.py
updated README.md

0.3.0 (2021-10-23)¶

masked pyslk.slk_search and pyslk.slk_group because they do not exist in the slk version for public release (will come later with an update)
updated descriptions
code layout updates after flake8 checks
basic structure for doc generation

0.2.2 (2021-10-02)¶

new function: parsers.slk_exists_bool()

0.2.1 (2021-10-02)¶

new functions: parsers.valid_token() and parsers.valid_session() (synonyms)

0.2.0 (2021-10-01)¶

package setup:
- setup.py, setup.cfg
- requirements*.txt files
- LICENCE
- README.md
finalized all slk functions (group, owner, chmod, tag)
split one large source file into several modules
updates in some function headers (pyslk.slk_delete, pyslk.slk_tag)
several minor bug fixes

0.1.0 (2021-09-20)¶

Initial Release