librarian v0.1.2 SSH View Source

SSH streams and SSH and basic SCP functionality

The librarian SSH module provides SSH streams (see stream/3) and three protocols over the SSH stream:

  • run/3, which runs a command on the remote SSH host.
  • fetch/3, which uses the SCP protocol to obtain a remote file,
  • send/4, which uses the SCP protocol to send a file to the remote host.

Note that not all SSH hosts (for example, embedded shells), implement an SCP command, so you may not necessarily be able to perform SCP over your SSH stream.

Using SSH

The principles of this library are simple. You will first want to create an SSH connection using the connect/2 function. There you will provide credentials (or let the system figure out the default credentials). The returned conn term can then be passed to the multiple utilities.

{:ok, conn} = SSH.connect("some.other.server")
SSH.run!(conn, "echo hello ssh")  # ==> "hello ssh"

Bang vs non-bang functions

As a general rule, if you expect to run a single or series of tasks with transient (or no) supervision, for example in a worker task or elixir script you should use the bang function and let the task fail, designing your supervision accordingly. This will also potentially let you be lazy about system resources such as SSH connections.

If you expect your SSH task to run as a part of a long-running process (for example, checking in on a host and retrieving data), you should use the error tuple forms and also be careful about closing your ssh connections after use. Check the connection labels documentation for a strategy to organize your code around this neatly.

Mocking

There's a good chance you'll want to mock your SSH commands and responses. The SSH.Api behaviour module is provided for that purpose.

Logging

The SSH and related modules interface with Elixir (and Erlang's) logging facility. The default metadata tagged on the message is ssh: true; if you would like to set it otherwise you can set the :librarian, :ssh_metadata application environment variable.

Customization

If you would like to write your own SSH stream handlers that plug in to the SSH stream and provide either rudimentary interactivity or early stream token processing, you may want to consider implementing a module following the SSH.ModuleApi behaviour, and initiating your stream as desired.

Limitations

This library has largely been tested against Linux SSH clients. Not all SSH schemes are amenable to stream processing. In those cases you should implement an ssh client gen_server using erlang's ssh_client, though support for this in elixir is planned in the near-term.

Link to this section Summary

Types

channel reference for the SSH and SCP operations

connection reference for the SSH and SCP operations

erlang ip4 format, {byte, byte, byte, byte}

connect to a remote is specified using either a domain name or an ip address

unix-style return codes for ssh-executed functions

Functions

closes the ssh connection.

initiates an ssh connection with a remote server.

like connect/2 but raises with a ConnectionError instead of emitting an error tuple.

retrieves a binary file from the remote host.

like fetch/3 except raises instead of emitting an error tuple.

runs a command on the remote host.

like run/3 except raises on errors instead of returning an error tuple.

sends binary content to the remote host.

like send/4, except raises on errors, instead of returning an error tuple.

creates an SSH stream struct as an ok tuple or error tuple.

like stream/2, except raises on an error instead of an error tuple.

Link to this section Types

channel reference for the SSH and SCP operations

connection reference for the SSH and SCP operations

erlang ip4 format, {byte, byte, byte, byte}

connect to a remote is specified using either a domain name or an ip address

Link to this type

retval()

View Source
retval() :: 0..255

unix-style return codes for ssh-executed functions

Link to this section Functions

Link to this function

close(conn)

View Source
close(conn() | term()) :: :ok | {:error, String.t()}

closes the ssh connection.

Typically you will pass the connection reference to this function. If your connection is contained to its own transient task process, you may not need to call this function as the ssh client library will detect that the process has ended and clean up after you.

In some cases, you may want to be able to close a connection out-of-band. In this case, you may label your connection and use the label to perform the close operation. See labels

Link to this function

connect(remote, options \\ [])

View Source
connect(remote(), keyword()) :: connect_result()

initiates an ssh connection with a remote server.

options:

  • :use_ssh_config see SSH.Config, defaults to false.
  • :global_config_path see SSH.Config.
  • :user_config_path see SSH.Config.
  • :user username to log in as.
  • :port port to use to ssh, defaults to 22.
  • :label see labels

and other SSH options. Some conversions between ssh options and SSH.connect options:

ssh commandline optionSSH library option
-o StrictHostKeyChecking=nosilently_accept_hosts: true
-qquiet_mode: true
-o ConnectTimeout=timeconnect_timeout: time_in_ms
-i pemfileidentity: file

also consult documentation on client options in the erlang docs

labels:

You can label your ssh connection to provide a side-channel for correctly closing the connection pid. This is most useful in the context of with/1 blocks. As an example, the following code works:

def run_ssh_tasks do
  with {:ok, conn} <- SSH.connect("some_host", label: :this_task),
       {:ok, _result1, 0} <- SSH.run(conn, "some_command"),
       {:ok, result2, 0} <- SSH.run(conn, "some other command") do
    {:ok, result1}
  end
after
  SSH.close(:this_task)
end

Some important points:

  • If you are wrangling multiple SSH sessions, please use unique connection labels.
  • The ssh connection label is stored in the process dictionary, so the label will not be valid across process boundaries.
  • If the ssh connection failed in the first place, the tagged close will return an error tuple. In the example, this will be silent.
Link to this function

connect!(remote, options \\ [])

View Source
connect!(remote(), keyword()) :: conn() | no_return()

like connect/2 but raises with a ConnectionError instead of emitting an error tuple.

Link to this function

fetch(conn, remote_file, options \\ [])

View Source
fetch(conn(), Path.t(), keyword()) :: fetch_result()

retrieves a binary file from the remote host.

Under the hood, this uses the scp protocol to transfer files.

The SCP protocol is as follows:

  • execute scp remotely in the undocumented -f <source> mode
  • send a single zero byte to initiate the conversation
  • wait for a control string "C0<perms> <size> <filename>"
  • send a single zero byte
  • wait for the binary data + terminating zero
  • send a single zero byte

The perms term should be in octal, and the filename should be rootless.

Example:

SSH.fetch(conn, "path/to/desired/file")
Link to this function

fetch!(conn, remote_file, options \\ [])

View Source
fetch!(conn(), Path.t(), keyword()) :: binary() | no_return()

like fetch/3 except raises instead of emitting an error tuple.

Link to this function

run(conn, cmd, options \\ [])

View Source
run(conn(), String.t(), keyword()) :: run_result()

runs a command on the remote host.

Options

  • {iostream, redirect}: iostream may be either :stdout or :stderr. redirect may be one of the following:

    • :stream sends the data to the stream.
    • :stdout sends the data to the group_leader stdout.
    • :stderr sends the data to the standard error io stream.
    • :silent deletes all of the data.
    • :raw sends the data to the stream tagged with source information as either {:stdout, data} or {:stderr, data}, as appropriate.
    • {:file, path} sends the data to a new or existing file at the provided path.
    • fun/1 processes the data via the function, with the output flat-mapped into the stream. this means that the results of fun/1 should be lists, with an empty list sending nothing into the stream.
    • fun/2 is like fun/1 except the stream struct is passed as the second parameter. The output of fun/2 should take the shape {flat_map_results, modified_stream}. You may use the :data field of the stream struct to store arbitrary data; and a value of nil indicates that it has been unused.
  • {:dir, path}: changes directory to path and then runs the command
  • {:as, :binary} (default): outputs result as a binary
  • {:as, :iolist}: outputs result as an iolist
  • {:as, :tuple}: result takes the shape of the tuple {stdout_binary, stderr_binary} note that this mode will override any other redirection selected.

Example:

SSH.run(conn, "hostname")  # ==> {:ok, "hostname_of_remote\n", 0}

SSH.run(conn, "some_program", stderr: :silent) # ==> similar to running "some_program 2>/dev/null"

SSH.run(conn, "some_program", stderr: :stream) # ==> similar to running "some_program 2>&1"

SSH.run(conn, "some_program", stdout: :silent, stderr: :stream) # ==> only capture standard error
Link to this function

run!(conn, cmd, options \\ [])

View Source
run!(conn(), String.t(), keyword()) :: run_content() | no_return()

like run/3 except raises on errors instead of returning an error tuple.

Note that by default this raises in the case that the SSH connection fails AND in the case that the remote command returns non-zero.

Link to this function

send(conn, content, remote_file, options \\ [])

View Source
send(conn(), iodata(), Path.t(), keyword()) :: send_result()

sends binary content to the remote host.

Under the hood, this uses the scp protocol to transfer files.

Protocol is as follows:

  • execute scp remotely in the undocumented -t <destination> mode
  • send a control string "C0<perms> <size> <filename>"
  • wait for single zero byte
  • send the binary data + terminating zero
  • wait for single zero byte
  • send EOF

The perms term should be in octal, and the filename should be rootless.

options:

  • :permissions - sets unix-style permissions on the file. Defaults to 0o644

Example:

SSH.send(conn, "foo", "path/to/desired/file")
Link to this function

send!(conn, content, remote_file, options \\ [])

View Source
send!(conn(), iodata(), Path.t(), keyword()) :: :ok | no_return()

like send/4, except raises on errors, instead of returning an error tuple.

Link to this function

stream(conn, cmd, options \\ [])

View Source
stream(conn(), String.t(), keyword()) ::
  {:ok, SSH.Stream.t()} | {:error, String.t()}

creates an SSH stream struct as an ok tuple or error tuple.

Options

  • {iostream, redirect}: iostream may be either :stdout or :stderr. redirect may be one of the following:

    • :stream sends the data to the stream.
    • :stdout sends the data to the group_leader stdout.
    • :stderr sends the data to the standard error io stream.
    • :silent deletes all of the data.
    • :raw sends the data to the stream tagged with source information as either {:stdout, data} or {:stderr, data}, as appropriate.
    • {:file, path} sends the data to a new or existing file at the provided path.
    • fun/1 processes the data via the function, with the output flat-mapped into the stream. this means that the results of fun/1 should be lists, with an empty list sending nothing into the stream.
    • fun/2 is like fun/1 except the stream struct is passed as the second parameter. The output of fun/2 should take the shape {flat_map_results, modified_stream}. You may use the :data field of the stream struct to store arbitrary data; and a value of nil indicates that it has been unused.
  • {:stream_control_messages, boolean}: should the stream control messages :eof, or {:retval, integer} be sent to the stream?
  • module: {mod, init}, The stream is operated using an module with behaviour SSH.ModuleApi
  • data_timeout: timeout, how long to wait between packets till we send a timeout event.
Link to this function

stream!(conn, cmd, options \\ [])

View Source
stream!(conn(), String.t(), keyword()) :: SSH.Stream.t() | no_return()

like stream/2, except raises on an error instead of an error tuple.