######################### Guide to python-simplezfs ######################### Overview ******** The interfaces are similar to calling the zfs toolset on the command line. That is, there are not state holding classes representing filesets or pools, each call must include the entire dataset path or pool name. There is also no way to collect actions and run them at the end, each action is carried out immediately. There are, however, two implementations of the functionality, using the ``cli`` tools and another one using the `libzfs_core` ``native`` library. We'll focus on the ``cli`` version here. Most of the functions raise a :class:`simplezfs.exceptions.ValidationError` with some helpful text if any of the data turns out not to be valid. For example, including invalid or reserved strings in the dataset name raises this exception. Let's take a look at the two interfaces **ZFS** and **ZPool**... .. warning:: All of the commands here attempt to modify something in the pool or dataset given as parameters. If run with high enough permission (usually ``root``, but there's ``zfs allow`` that can delegate to lower- privileged users, too) these commands can and will **delete** data! Always run these against pools, disks and datasets that bear no important data! You have been warned. The ZFS interface ***************** The :class:`simplezfs.zfs.ZFS` is the interface that corresponds to the ``zfs(8)`` program. It holds very little state, and it is recommended to get an instance through the function :func:`~simplezfs.zfs.get_zfs`. It selects the desired implementation and passes the required parameters. At the very least, it requires the ``api``-parameter, which is a string that selects the actual implementation, either ``cli`` or ``native``. All our examples use the ``cli`` implementation for simplicity. .. code-block:: pycon >>> from zfs import get_zfs >>> zfs = get_zfs('cli') >>> zfs For the remainder of this guide, we're going to assume that the variable ``zfs`` always holds a :class:`simplezfs.zfs_cli.ZFSCli` object. Viewing data ============ To get an overview over the interface, we'll dive in and inspect our running system. Information is returned in the form if :class:`simplezfs.types.Dataset` instances, which is a named tuple containing a set of fields. For simplicity, we'll output only a few of its fields to not clobber the screen, so don't be alarmed if there seems to be information missing: we just omitted the boring parts. Listing datasets ---------------- By default when listing datasets, all of them are returned regardless of their type. That means that it includes * volumes * filesets * snapshots * bookmars .. code-block:: pycon >>> zfs.list_datasets() This is often unneccessary, and it allows to limit both by ``type`` and by including only datasets that are children of another, and both at the same time: .. code-block:: pycon >>> zfs.list_datasets(type=DatasetType.SNAPSHOT) >>> zfs.list_datasets(parent='pool/system') >>> zfs.list_datasets(parent='pool/system', type=DatasetType.SNAPSHOT) Creating something new ====================== There are functions for creating the four different types of datasets with nice interfaces: * :func:`~simplezfs.zfs.ZFS.create_fileset` for ordinary filesets, the most commonly used parameter is ``mountpoint`` for telling it where it should be mounted. * :func:`~simplezfs.zfs.ZFS.create_volume` creates volumes, or ZVols, this features a parameter ``thin`` for creating thin-provisioned or sparse volumes. * :func:`~simplezfs.zfs.ZFS.create_snapshot` creates a snapshot on a volume or fileset. * :func:`~simplezfs.zfs.ZFS.create_bookmark` creates a bookmark (on recent versions of ZFS). These essentially call :func:`~simplezfs.zfs.ZFS.create_dataset`, which can be called directly, but its interface is not as nice as the special purpose create functions. Filesets -------- Creating a fileset requires the dataset path, like this: .. code-block:: pycon >>> zfs.create_fileset('pool/test', mountpoint='/tmp/test') :todo: add create_dataset Volumes ------- Volumes are created similar to filesets, this example creates a thin-provisioned sparse volume: .. code-block:: pycon >>> zfs.create_volume('pool/vol', thin=True) :todo: add create_dataset Snapshots --------- Snapshots are, like bookmarks, created on an existing fileset or volume, hence the first parameter to the function is the dataset that is our base, and the second parameter is the name of the snapshot. .. code-block:: pycon >>> zfs.create_snapshot('pool/test', 'pre-distupgrade') Bookmarks --------- Like snapshots above, bookmarks are created on an existing fileset or volume. .. code-block:: pycon >>> zfs.create_bookmark('pool/test', 'book-20190723') Destroying things ================= After creating some datasets of various kinds and playing around with some of their properties, it's time to clean up. We'll use the ``destroy_*`` family of methods. .. warning:: Bear in mind that things happening here are final and cannot be undone. When playing around, always make sure not to run this on pools containing important data! Filesets -------- Volumes ------- Snapshots --------- Bookmarks --------- Properties ========== Properties are one of the many cool and useful features of ZFS. They control its behaviour (like ``compression``) or return information about the internal states (like ``creation`` time). .. note:: The python library does not validate the names of native properties, as these are subject to change with the ZFS version and it would mean that the library needs an update every time a new ZFS version changes some of these. Thus, it relies on validating the input for syntax based on the ZFS documentation of the OpenZFS project and ZFS telling it that it did not like a name. A word on metadata/user properties ---------------------------------- The API allows to get and set properties, for both ``native`` properties (the ones defined by ZFS, exposing information or altering how it works) and ``user`` properties that we call **metadata properties** in the API. When working with metadata properties, you need to supply a ``namespace`` to distinguish it from a native property. This works by separating the namespace and the property name using a ``:`` character, so a property ``myprop`` in the namespace ``com.company.department`` becomes ``com.company.department:myprop`` in the ZFS property system. This is done automatically for you if you supply a ``metadata_namespace`` when creating the ZFS instance and can be overwritten when working with the get and set functions. It is also possible not to define the namespace and passing it to the functions every time. When you want to get or set a metadata property, set ``metadata`` to **True** when calling :func:`~simplezfs.zfs.ZFS.get_property` or :func:`~simplezfs.zfs.ZFS.set_property`. This will cause it to automatically prepend the namespace given on instantiation or to prepend the one given in the ``overwrite_metadata_namespace`` when calling the functions. The name of the property **must not** include the namespace, though it may contain ``:`` characters on its own, properties of the form ``zfs:is:cool`` are valid afterall. ``:`` characters are never valid in the context of native properties, and this is the reason why there is a separate switch to turn on metadata properties when using these functions. Error handling -------------- If a property name is not valid or the value exceeds certain bounds, a :class:`simplezfs.exceptions.ValidationError` is raised. This includes specifying a namespace in the property name if ``metadata`` is **False**, or exceeding the length allowed for a metadata property (8192 - 1 bytes). Though not an error for the ``zfs(8)`` utility, getting a non-existing metadata property also raises the above exception to indicate that the property does not exist. Getting a property ------------------ Getting properties is fairly straight-forward, especially for native properties: .. code-block:: pycon >>> zfs.get_property('tank/system/root', 'mountpoint') Property(key='mountpoint', value='/', source='local', namespace=None) For **metadata** properties, one needs to enable their usage by setting ``metadata`` to True. With a globally saved namespace, it looks like this: .. code-block:: pycon >>> zfs = get_zfs('cli', metadata_namespace='com.company') >>> zfs.get_property('tank/system/root', 'do_backup', metdata=True) Property(key='do_backup', value='true', source='local', namespace='com.company') If you don't specify a namespace when calling :func:`~simplezfs.zfs.get_zfs` or if you want to use a different namespace for one call, specify the desired namespace in ``overwrite_metadata_namespace`` like so: .. code-block:: pycon >>> zfs.get_property('tank/system/root', 'requires', metadata=True, overwrite_metadata_namespace='user') Property(key='requires', value='coffee', source='local', namespace='user') This is the equivalent of calling ``zfs get user:requires tank/system/root`` on the shell. Asking it to get a native property that does not exist results in an error: .. code-block:: pycon >>> zfs.get_property('tank/system/root', 'notexisting', metadata=False) zfs.exceptions.PropertyNotFound: invalid property on dataset tank/test Setting a property ------------------ The interface for setting both native and metadata properties works exactly like the get interface shown earlier, though it obviously needs a value to set. We won't go into ZFS delegation system (``zfs allow``) and assume the following is run using **root** privileges. .. code-block:: pycon >>> zfs.set_property('tank/service/backup', 'mountpoint', ''/backup') Setting a metadata property works like this (again, like above): .. code-block:: pycon >>> zfs.set_property('tank/system/root', 'requires', 'tea', metadata=True, overwrite_metadata_namespace='user') Listing properties ------------------ :todo: ``zfs.get_properties`` The ZPool interface ******************* The :class:`simplezfs.zfs.ZPool` is the interface that corresponds to the ``zpool(8)`` program. It holds very little state, and it is recommended to get an instance through the function :func:`~simplezfs.zpool.get_zpool`. It selects the desired implementation and passes the required parameters. At the very least, it requires the ``api``-parameter, which is a string that selects the actual implementation, either ``cli`` or ``native``. All our examples use the ``cli`` implementation for simplicity. .. code-block:: pycon >>> from simplezfs import get_zpool >>> zpool = get_zpool('cli') >>> zpool For the remainder of this guide, we're going to assume that the variable ``zpool`` always holds a :class:`simplezfs.zpool_cli.ZPoolCli` object. Error handling ************** We kept the most important part for last: handling errors. The module defines its own hierarchy with :class:`simplezfs.exceptions.ZFSException` as toplevel exception. Various specific exceptions are based on ot. When working with :class:`simplezfs.zfs.ZFS`, the three most common ones are: * :class:`simplezfs.exceptions.ValidationError` which indicates that a name (e.g. dataset name) was invalid. * :class:`simplezfs.exceptions.DatasetNotFound` is, like FileNotFound in standard python, indicating that the dataset the module was instructed to work on (e.g. get/set properties, destroy) was not present. * :class:`simplezfs.exceptions.PermissionError` is raised when the current users permissions are not sufficient to perform the requested operation. While some actions can be delegated using ``zfs allow``, linux, for example, doesn't allow non-root users to mount filesystems, which means that a non-root user may create filesets with a valid mountpoint property, but it won't be mounted. Examples ======== .. code-block:: pycon >>> zfs.list_dataset(parent=':pool/name/invalid') zfs.exceptions.ValidationError: malformed name .. code-block:: pycon >>> zfs.list_datasets(parent='pool/not/existing') zfs.exceptions.DatasetNotFound: Dataset "pool/not/existing" not found