spars

Spars is a commandline tool to query the
spark-history daemon on info about applications.

Spars commands

The two major sub-commands are list and show, as in:

spars list
spars show

spars list lists general information in lists
for applications, jobs and stages.

spars show shows detailed information on a
specific application, job or stage.

To see filtering flags and other positional arguments, please use

spars --help

# spars <sub-command> --help
spars list --help

Spars config

Default config is to connect to "localhost" on default port 18080.
To connect to other host or to use a non default port,
create a config file in one of the following locations:

• $HOME/.spars/spars.conf
• /etc/spars/spars.conf

The config file can contain one or both
of the following parameters:

host = "<hostname or IP address as a string>"
port = <port as unsigned integer>

If one of the parameters is not specified the default
value will be used.

Spark-history

Spark-history daemon has to be configured to read
info from a specific directory an clients need to
be configured to write logs to the same directory,
for the information to be available in the
spark-history daemon.

Following parameters has to be configured in:
spark-defaults.conf

# spark.eventLog.dir: the directory where clients write logs,
# should be the same as where history daemon reads
spark.eventLog.enabled true
spark.eventLog.dir /dir/where/history/daemon/reads
spark.history.fs.logDirectory /dir/where/history/daemon/reads