Introducing Abscissa: iqlusion’s security-oriented Rust application framework
By Tony Arcieri
Earlier this month we released Abscissa: our security-oriented Rust application framework. After releasing v0.1, we’ve spent the past few weeks further polishing it up in tandem with this blog post, and just released a follow-up v0.2. The Abscissa source repository is hosted on GitHub.
Here at iqlusion we have developed a number of Rust applications, ranging from CLI applications (e.g. devops tools) to network services including the Tendermint Key Management System. Many of these applications used a common set of crate dependencies and copy/paste boilerplate. After a lot of work, we have moved (or are in the process of moving) all of these tools to Abscissa.
Don’t let the “version 0.2” fool you: the codebase that would become Abscissa is now a little over a year old. We have deliberately left the version number in the “0.0.x” range because much of it started as ugly copy-paste boilerplate which wasn’t really ready for public consumption However, over the past several months we’ve spent a lot of time cleaning up the codebase, adding an easy-to-use code generator, backfilling tests, and shaping it into something that is both ready for public consumption and something we plan to use immediately as the basis for many more new Rust applications.
At the same time, we’re at v0.2 for a reason: it is still early days for Abscissa and we are not yet close to a “stable” release. As people who have been using Rust prior to 1.0, we believe Rust and its litany of compile-time checks makes it possible to evolve codebases built on an unstable foundation, and so we’re excited to keep evolving Abscissa based on real-world pain points. However, if you’re looking for a smooth ride, it might be a bit early for you and Abscissa is something you should keep on your radar.
What is Abscissa? #
Abscissa is a Rust application framework. We call it that because it falls into a family of frameworks which generate a small application skeleton for you from an initial template. We’ve taken care to minimize the boilerplate to allow us to evolve the framework while minimizing boilerplate modifications, however at least for Abscissa 0.x we expect each release to involve some minor boilerplate changes.
While we think Abscissa is of general interest to the Rust community, we’ve built it to fill a particular niche: high-security applications which run in mission critical contexts. We value security above everything else, including things like performance, using the best available ecosystem crates when they have an onerous number of transitive dependencies, and at least for now things like portability and internationalization. That’s not to say we don’t care about the latter, but as we work on it we will carefully weigh proposed solutions against the potential risks posed by upstream dependencies.
The “Tier 1” platform for Abscissa is Linux, but we also test on Windows and macOS. We primarily intend for Abscissa to be used to make CLI applications and network services that run on Linux, but are also macOS users, and also want to keep the door open to potentially run our applications on Windows in the future.
Abscissa provides the following features (the following is copy/paste from the Abscissa documentation):
- command-line option parsing: simple declarative option parser based on gumdrop. The option parser in Abcissa contains numerous improvements which provide better UX and tighter integration with the other parts of the framework (e.g. overriding configuration settings using command-line options).
- components: Abscissa uses a component architecture (similar to an ECS) for extensibility/composability, with a minimalist implementation that still provides such features such as calculating dependency ordering and providing hooks into the application lifecycle. Newly generated apps use two components by default:
terminal
andlogging
. - configuration: Simple parsing of TOML configurations to serde-parsed configuration types which can be dynamically updated at runtime.
- error handling: generic
Error
type based on thefailure
crate, and a unified error-handling subsystem. - logging: based on the
log
to provide application-level logging. - secrets management: the optional
Secret
type from oursecrecy
crate impls serde’sDeserialize
and can be used to represent secret values parsed from configuration files or elsewhere (e.g. credentials loaded from the environment or network requests). - terminal interactions: support for colored terminal output (with color support autodetection). Useful for Cargo-like status messages with easy-to-use macros.
Despite all of this functionality, we’ve tried to keep Abscissa’s actual code footprint (both in terms of implementation size and number of dependencies) as small as we possibly can: Abscissa itself is presently ~3 klocs, and with all features enabled has ~30 transitive dependencies. We hope most of these dependencies have familiar names: backtrace
, chrono
, failure
, lazy_static
, libc
, log
. We’ll talk more about our dependency strategy later.
What projects are using Abscissa? #
We’ve been using Abscissa for some time in a number of applications, and have recently ported others (developed using the original copy/paste boilerplate Abscissa was based on) over to it as well. Here are a quick list of noteworthy projects using Abscissa:
- Tendermint KMS: key management system for Tendermint blockchain applications.
- cargo-audit: auditing tool for the RustSec security advisory database.
- Zcash Foundation’s
zebra
: Rust implementation of a Zcash node. - Various iqlusion tooling: we are using Abscissa for a number of other tools we are developing, such as the canister deployer and sear encrypted archiving utility.
Abscissa isn’t a framework we created just for the sake of creating a framework: we created it because we were spending too much time maintaining copy/paste boilerplate, and because we had a real itch to scratch. We are committed to using it, so as long as we are around, it will be too. As David Heinemeier Hansson says, “Great frameworks are extractions, not inventions”
Inspiration #
Abscissa’s two main influences are Dropwizard and Ruby on Rails. Though both of those are web frameworks and Abscissa is not, web application frameworks end up solving a lot of the same problems.
Dropwizard took many top ecosystem libraries from Java and assembled them into an application framework. It provides an Application base class and Configuration base class which conceptually inspired similar traits in Abscissa.
Ruby on Rails is known for many things. The main ones we’ve adopted in Abscissa are an application generator and leveraging custom derive to eliminate boilerplate code. Additionally, there are many philosophical aspects to the project we seek to emulate: Abscissa is an “omakase” framework which seeks to provide a single way of doing things, a.k.a. “convention over configuration”.
Creating an Abscissa Application #
Abscissa uses an application boilerplate generator, similar to cargo new
(or e.g. rails new
) to create an application skeleton. To install this boilerplate generator, run the following:
$ cargo install abscissa
This will compile and install the abscissa
CLI command. Note that the abscissa
crate contains the CLI application (itself a full-blown Abscissa application), while the framework-level functionality is all found in the abscissa_core
crate.
This command’s main notable functionality is the abscissa new
command, which you can use to create an application:
Now that you’ve created an application, let’s take a quick tour of its parts.
Tour of an Abscissa Application #
Every Abscissa application consists of the following parts:
main.rs
: Rust binary entrypointApplication
type: application-defined type which owns and synchronizes access to all application stateConfig
type: application configuration (loaded from a TOML file)Command
types: CLI (sub)commands and argument parsersError
type: application-specific error types- Acceptance tests: exercise CLI apps to ensure they behave correctly
In this section I’ll briefly describe each of these.
main.rs
: Rust binary entrypoint #
Every Abscissa application created with the generator is a Rust package containing two crates:
lib
: your actual application codebin
: a thin wrapper for launching your application
Abscissa applications are Rust libraries. One of our eventual goals is to allow several Abscissa applications to coexist in the same binary, allowing for things like embedding subcommands/subapplications.
The main.rs
file is located under src/bin/<your_app>/main.rs
and presently contains the following:
//! Main entry point for My Cool App
#![deny(warnings, missing_docs, trivial_casts, unused_qualifications)]
#![forbid(unsafe_code)]
use my_cool_app::application::APPLICATION;
/// Boot My Cool App
fn main() {
abscissa_core::boot(&APPLICATION);
}
The APPLICATION
constant (see more about applications below) is a lazy_static
containing your entire application state, which initializes to a default “unbooted” state. As you can see, the main.rs
shim is about as simple as it possibly can be given this sort of application structure.
Application
: Core application state and lifecycle management #
The heart of any Abscissa application is a user-defined type located in src/application.rs
which impls the abscissa_core::Application
trait and owns all of the application state. It borrows some ideas from Entity-Component-System patterns in Rust as described in Catherine West’s RustConf 2018 closing keynote (that said, we do NOT consider Abscissa to be an “ECS”, and it presently lacks many of the interesting features described in the talk like generational index allocators).
Here are some excerpts from the generated src/application.rs
file:
lazy_static! {
/// Application state
pub static ref APPLICATION: Lock<MyApplication> = Lock::default();
}
/// My Application
#[derive(Debug)]
pub struct MyApplication {
/// Application configuration.
config: Option<MyConfig>,
/// Application state.
state: application::State<Self>,
}
/// Initialize a new application instance.
///
/// By default no configuration is loaded, and the framework state is
/// initialized to a default, empty state (no components, threads, etc).
impl Default for MyCoolAppApplication {
fn default() -> Self {
Self {
config: None,
state: application::State::default(),
}
}
}
Applications presently consist of their configuration (see more on Config
below), and a framework-managed abscissa_core::application::State
type. However, this struct is your application type, and you’re free to add anything else to it which makes sense. You can customize the defaults as you see fit, e.g. starting with a default configuration rather than None
, for example (a change we are considering making upstream).
Access to the application is guarded by abscissa_core::application::lock::Lock, which is a wrapper for a RwLock. This allows for concurrent read-only access to application state, with exclusive access for mutation.
Additionally, the default boilerplate generates a set of application state accessors (re-exported through your application’s crate::prelude::*
) for obtaining locks on this state:
/// Obtain a read-only (multi-reader) lock on the application state.
pub fn app_reader() -> application::lock::Reader<MyCoolAppApplication> {
APPLICATION.read()
}
/// Obtain an exclusive mutable lock on the application state.
pub fn app_writer() -> application::lock::Writer<MyCoolAppApplication> {
APPLICATION.write()
}
/// Obtain a read-only (multi-reader) lock on the application configuration.
///
/// Panics if the application configuration has not been loaded.
pub fn app_config() -> config::Reader<MyCoolAppApplication> {
config::Reader::new(&APPLICATION)
}
All that said, abscissa_core::Application
is a complex trait which manages much of the application lifecycle, and describing everything it does is a bit much for a whirlwind tour of the framework. For more information, check out the linked rustdoc, and stay tuned for future blog posts about how Abscissa handles the application lifecycle.
Config
: Configuration file support #
Abscissa includes built-in support for loading TOML configuration files using serde and the toml
crate, which are parsed into a user-defined configuration type located in src/config.rs
which impls the abscissa_core::Config trait. Here is the boilerplate which will appear in Abscissa v0.2 (you will get slightly different boilerplate if you use v0.1):
/// My Configuration
#[derive(Clone, Config, Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct MyConfig {
/// An example configuration section
pub hello: ExampleSection,
}
impl Default for MyConfig {
fn default() -> Self {
Self {
hello: ExampleSection::default(),
}
}
}
/// Example Config Section
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct ExampleSection {
/// Example configuration value
pub recipient: String,
}
impl Default for ExampleSection {
fn default() -> Self {
Self {
recipient: "world".to_owned(),
}
}
}
Configuration is loaded automatically at the time the application starts and can be accessed by calling the app_config()
helper from the boilerplate above, which obtains a read-only mutex guard for the application and makes the configuration available via Deref
.
This helper is automatically exported through the applicaiton-local prelude. With a use crate::prelude::*
at the top of a module in your application, accessing configuration as easy as calling:
let config = app_config();
Command
types: CLI application subcommands #
Abscissa is written from the ground-up to create CLI utilities with multiple subcommands. It uses gumdrop as the option parser, which like structopt
uses a proc macro-based “DSL” for generating declarative command-line option parsers. Application subcommands are found in src/commands*
and must impl the abscissa_core::Command
trait, which can be derived automatically using the abscissa_derive
crate and derive(Command)
.
The boilerplate includes a toplevel command which looks like the following:
/// My Subcommands
#[derive(Command, Debug, Options, Runnable)]
pub enum MyCommand {
/// The `help` subcommand
#[options(help = "get usage information")]
Help(Help<Self>),
/// The `start` subcommand
#[options(help = "start the application")]
Start(StartCommand),
/// The `version` subcommand
#[options(help = "display version information")]
Version(VersionCommand),
}
Similarly to structopt
, these enum variants are parsed as the help
, start
, and version
subcommands respectively. The StartCommand
and VersionCommand
structs are included in the boilerplate, however note that the Help
subcommand is generic around another Command
and defined in Abscissa as abscissa_core::command::Help
.
Note that MyCommand
is doing derive(Runnable)
. The abscissa_core::Runnable
trait (similar to the Java interface with the same name, for those familiar with it) provides an associated ::run
method for each command, and can be derived on any enum
so as to invoke ::run
on the appropriate variants (see below for a Runnable
impl).
Below is the definition of StartCommand
included in the boilerplate:
#[derive(Command, Debug, Options)]
pub struct StartCommand {
/// To whom are we saying hello?
#[options(free)]
recipient: Vec<String>,
}
impl Runnable for StartCommand {
/// Print "Hello, world!"
fn run(&self) {
if self.recipient.is_empty() {
println!("Hello, world!");
} else {
println!("Hello, {}!", self.recipient.join(" "));
}
}
}
The above shows an example of gumdrop
argument parsing, namely the start
command can take optional “free” arguments, e.g. myapp start everybody
would print “Hello, everybody!”
Backing up to the toplevel MyCommand
enum, there’s one additional noteworthy trait impl that comes in the boilerplate (below is using boilerplate from the unreleased Abscissa v0.2):
/// Configuration filename
pub const CONFIG_FILE: &str = "myapp.toml";
/// This trait allows you to define how application configuration is loaded.
impl Configurable<MyConfig> for MyCommand {
/// Location of the configuration file
fn config_path(&self) -> Option<PathBuf> {
Some(PathBuf::from(CONFIG_FILE))
}
/// Apply changes to the config after it's been loaded, e.g. overriding
/// values in a config file using command-line options.
fn process_config(&self, config: MyConfig) -> Result<MyConfig, FrameworkError> {
match self {
MyCommand::Start(cmd) => cmd.override_config(config),
_ => Ok(config),
}
}
}
The abscissa_core::Configurable
trait controls both the path from which the TOML file containing the configuration will be loaded, and also allows preprocessing the raw configuration data before it is stored within the application state. The latter is useful for things like overriding configuration settings using command-line options.
Error
types #
Abscissa provides an abscissa_core::Error
type which is generic around a Kind
(intended to be an enum of possible error variants). Abscissa’s error types are based on the failure
crate and capture both a cause as well as backtraces.
Internally Abscissa leverages its own abscissa_core::error::FrameworkError
type as a concrete instantiation of this type, while also defining an application-specific newtype. Here is what the boilerplate for that looks like (located under src/error.rs
):
/// Error type
#[derive(Debug)]
pub struct Error(abscissa_core::Error<ErrorKind>);
/// Kinds of errors
#[derive(Copy, Clone, Eq, PartialEq, Debug, Fail)]
pub enum ErrorKind {
/// Error in configuration file
#[fail(display = "config error")]
Config,
/// Input/output error
#[fail(display = "I/O error")]
Io,
}
impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
self.0.fmt(f)
}
}
impl From<abscissa_core::Error<ErrorKind>> for Error {
fn from(other: abscissa_core::Error<ErrorKind>) -> Self {
Error(other)
}
}
impl From<io::Error> for Error {
fn from(other: io::Error) -> Self {
err!(ErrorKind::Io, other).into()
}
}
As you can see, the ErrorKind
variant is defined using failure_derive
‘s proc macro-based “DSL”. Additionally the boilerplate provides a handful of trait impls, with the intention that you define others as applicable.
Acceptance tests #
Abscissa includes its own acceptance testing system, intended to execute your application’s binary (or potentially multiple binaries if you have several in your project) and make assertions about things like the output and exit status.
lazy_static! {
/// Executes your application binary via `cargo run`.
///
/// Storing this value in a `lazy_static!` ensures that all
/// instances of the runner acquire a mutex when
/// executing commands and inspecting exit statuses,
/// serializing what would otherwise be multithreaded
/// invocations as `cargo test` executes tests in parallel.
pub static ref RUNNER: CmdRunner = CmdRunner::default();
}
#[test]
fn start_no_args() {
let mut runner = RUNNER.clone();
let mut cmd = runner.arg("start").capture_stdout().run();
cmd.stdout().expect_line("Hello, world!");
cmd.wait().unwrap().expect_success();
}
#[test]
fn start_with_args() {
let mut runner = RUNNER.clone();
let mut cmd = runner
.args(&["start", "acceptance", "test"])
.capture_stdout()
.run();
cmd.stdout().expect_line("Hello, acceptance test!");
cmd.wait().unwrap().expect_success();
}
The abscissa_core::testing::CmdRunner
type provides the main entrypoint for running acceptance tests. As you can see above, we made CmdRunner::default()
do what you probably want it to do: run the main binary of your application.
Additionally, when used as a lazy_static
above, the ::default()
settings will also acquire a mutex, ensuring that tests which invoke your application as a subprocess are run in serial (as ordinarily Rust tests are multithreaded and run concurrently).
The test runner provides built-in support for a number of assertions, such as exact contents of lines written to STDOUT or STDERR, regex matching on those lines, and process exit codes.
If you’ve ever tested a Rust command line application before, you’ve probably done some or all of the above using a half dozen crates (or more), and that’s not counting those crates transitive dependencies. The Abscissa acceptance testing support, on the other hand, is not only completely built into the framework (gated under the testing
feature, which is enabled per default in the boilerplate under dev-dependencies
), but it’s all implemented in a way that depends only on the standard library.
Component
: Abscissa’s framework extension system #
Abscissa borrows some ideas from Entity-Component-System design (though we insist that it is not an ECS and should not be described as such) - all Abscissa applications consist of a single type which owns the application state. One of the most important types contained with the application state are components, types which impl the abscissa_core::Component
trait, Abscissa’s application lifecycle-aware extension system.
By default Abscissa applications include two components: terminal
for managing STDOUT and STDERR streams and terminal colors, and logging
which uses terminal
to write log output (for consumption by systemd-journald
, docker logs
, etc). Even with two components we already have an interdependency: logging
depends on terminal
, and needs the latter to be started first.
Solving the problem of interdependencies between parts of an application and starting them in the correct order has a number of solutions, from Rails’ simple “initializer” system to complex dependency injection frameworks.
Abscissa takes a slightly different approach: it uses a runtime component::Registry
which is able to compute the correct startup order at runtime using dependency information provided by each component. It does this by performing a topological sort of registered components based on dependencies they express. And while that might sound complicated, the implementation of this approach is largely in terms of Rust’s PartialOrd
trait.
Why a runtime registry and not a compile-time one? We wanted to make component startup flexible so it can be customized by things like application configuration. This allows for building binaries which include components and subsystems which aren’t necessarily activated at runtime, but can be configured to if the user so desires. Abscissa will calculate the dependency ordering based on components which are registered at the time the application boots.
Components are generic around an abscissa_core::Application
type and as part of their after_config
callback has access to the application’s config type (as a concrete associated type). This allows for components to export their configuration needs as traits which the application config type must impl, allowing them to be used across multiple applications while still consuming values from the application config.
Beyond configuration, however, components are very much dynamic: by making them generic around the application, and only using application-specific associated types, components wind up being safe for use as trait objects, and are stored internally within the component registry as Box<dyn Component>
, using downcasting in cases where it’s desirable to obtain a concrete component type. Making all of this work relies on several new Rust features for supporting dynamic types and dispatch, and for that reason (and a few others) Abscissa requires 1.35+.
All of that said, we feel Abscissa’s components (and Rust’s support for dynamic programming) are still a little rough around the edges. One big showstopper right now is that components can’t obtain references to each other: where the framework does that right now it’s done in terms of lazy_static
values. We’d like to move past that to storing component state in the components themselves.
This is where further incorporating ideas from ECS design will be helpful: namely we would like to move the component registry to use a generational index allocator, so components can hold references to each other by index, and in doing so providing a graph of index-based references between components which would otherwise be disallowed by the borrow checker. Only then can we really say that Abscissa has reached feature parity with things like dependency injection frameworks - otherwise we’re not quite there yet. But these are approaches we think will work and are excited to explore in future versions of the framework.
Thoughts on Dependencies #
We view Cargo, crates.io, and the overall crate ecosystem as among Rust’s biggest strengths. Ignoring everything else about the language, Rust is one of the first systems programming languages with a package manager as sophisticated as Cargo. We think if you aren’t leveraging the crates.io ecosystem, you aren’t leveraging Rust to its fullest. Also, as we noted earlier, we also think one of the best things Dropwizard (one of Abscissa’s inspirations) did was combine a number of existing, well-done libraries into a single cohesive package.
That said, crate dependencies are something of a hotbutton item right now. There are multiple threads on the rust-internals
forum about expanding the standard library and how much functionality crates should provide - are smaller crates or larger crates preferable?
This is in addition to several other threads about adding some sort of permission/capability system for restricting what dependencies are allowed to do. The core concern here is an important one: software supply chain attacks, which are now occurring on a regular basis and often target “blockchain” applications like we are using Abscissa for.
Crate dependencies, and their security implications, are something I have spent considerable time thinking about. In 2014 I opened one of the first crates.io GitHub issues about authenticating crates and potentially leveraging The Update Framework for doing so, and following that up with a concrete proposal to do so which is a modified version of a similar proposal by withoutboats. I also proposed an “unsafe features” approach to restricting the capabilities of crate dependencies.
There are no easy answers here - only tradeoffs. Should crates be big or small? That depends! A crate is probably too small if it takes more code to import and use it than relevant functionality that it contains, but where do you draw the line? Crates are one of Rust’s core modeling dimensions, and packaging code in a set of small-but-related crates is one way to make up for some of the ways in which Rust’s module system and conditional compilation features are not quite as sophisticated as we would like them to be yet.
From a purely security perspective, there is something that does hold universally true, however: less attack surface is better. Setting aside the metrics by which we gauge it for a moment, the less potentially attacker-controlled code we depend on, the more difficult Abscissa will be to attack. We are not looking for “big crates” or “small crates” per say (though I think it’s safe to say we generally prefer one of the former to a swarm of the latter), but rather make case-by-case assessments based on due diligence and scrutiny.
We have taken time to familiarize ourselves with each of Abscissa dependencies, what they are, what they do, and who writes them. The Abscissa README.md contains a large section on its dependencies, which enumerates all of them, whether they’re runtime or compile time dependencies, who the authors are, what the licenses are, and even cross-reference them with Abscissa features so you can see what you can optionally disable. Based on a careful analysis of the Rust crate ecosystem, we have concluded that Alex Crichton is the biggest risk for a software supply chain attack (just kidding Alex, and thanks for all the software!)
Though it’s a library, and this may seem a bit weird for a library, we check in the Cargo.lock
file so when transitive dependencies change we get some visibility. Going down the road we may even consider using tools like cargo vendor (which is in the process of being integrated into Cargo proper) in order to check in the complete source code of all of our crate dependencies so we can review changes to them whenever they are updated.
Dependency case study: term
crate #
One specific example of something we’ve caught with this approach is the term
crate: suddenly it was pulling in some cryptography-related code (namely the argon2rs crate). While argon2rs
is a perfectly fine crate which we’d recommend if you need a password-based KDF, we want to be sure about the provenance of all of the cryptographic crates in our application, lest collaborators on our project think certain cryptographic crates are “allowed” simply because they’re pulled in as transitive dependencies.
So where was argon2rs
coming from? It was introduced as a transitive dependency by this commit which replaced usage of the deprecated env::home_dir()
function with the dirs
crate, which transitively pulls in argon2rs
through the dirs-sys
crate’s Redox bindings. All this code was trying to do was find the user’s home directory, and to do that this small change introduced dozens of new dependencies. For the record for future crates which might be wanting to do this, we think the best way of locating the user’s home directory (particularly in contexts like this one which don’t rely on things like XDG user directories) is the home
crate, which is used by Cargo and rustup.
In investigating this problem however, we discovered a bigger problem with the term
crate though: it’s unmaintained!. While many lament the “std
is where code goes to die” problem, there’s a corollary in terms of widely used foundational dependencies like term
going unmaintained.
Our reaction to this was to rip out term
and replace it with the termcolor
crate. We ended up liking termcolor
a lot better than term
: it’s does everything we were using term
for but is also higher level and has integrated locking support. Furthermore it’s a post-1.0 crate by a well-respected crate author (BurntSushi) who also develops several other dependencies we use.
The term
crate story gets a little bit more complicated though: a week ago someone volunteered to maintain it. While we applaud of people stepping up to maintain foundational crates like term
, and don’t want to say anything specifically wrong about this person in particular (we have no reason to believe they are actively malicious), it doesn’t change our decision (everything else aside we like termcolor
better than term
) and also view these sorts of change-in-maintainer events as a potential software supply chain risk.
Transferring ownership of a popular library to a malicious new “maintainer” was exactly what happened in the compromise of the JavaScript event-stream
library last year. Though we’ve already moved off of it, we think it’s worth paying attention to what ends up happening to the term
crate, as it seems to be a particularly noteworthy case of a crate with nearly ubiquitous usage in many important Rust tools (including Cargo) which is presently unmaintained.
We present this entire story as a case study in what we hope to do with Abscissa’s dependencies: leverage outstanding crates which do what we want, but also keep an eye on all of them to ensure they are maintained, not undergoing scary ownership changes, and are free of malicious payloads.
Rust needs more frameworks #
Rust’s philosophy of building out and investing in package management early as opposed to investing the same in a “batteries-included std
” has many benefits in terms of allowing competing solutions to exist as opposed to mandating a single one that stagnates, but it’s a double-edged sword with drawbacks such as potential software supply chain attacks against these third party libraries, and also another problem: a lack of cohesion. The difficulty of putting together the multitude of crate dependencies needed to build an application, and associated boilerplate, is a large part of the motivation for Abscissa.
In absence of a “batteries-included std
”, frameworks like Abscissa become much-needed points-of-cohesion where crate development can be shaped by real-world interactions with other crates. We would like to work with other Rust framework developers to come up with a set (or sets) of crates which are widely used by multiple frameworks which can mostly be agreed upon.
We have definitely been keeping an eye on the work of the Rust CLI working group and are interested in leveraging some of their crates, particularly around configuration.
Crates we didn’t use but are still awesome anyway #
Finally, we’d like to conclude this section with a shout out to a couple of awesome Rust ecosystem crates we think are great, but didn’t use, and why we didn’t use them:
- cargo-generate is a fantastic application boilerplate generator, and one we’d recommend using if you wind up using something other than Abscissa. If it’s so great, why didn’t we use it? Well, we certainly tried! There are a few things we didn’t like about it for Abscissa’s purposes: namely it requires that the application boilerplate be kept in a separate git repo, whereas with Abscissa application generation ended up being critical to the way we make and test changes to the framework. We looked at various ways of handling separate repos for this purpose, like git submodules, but these complicate making atomic changes that affect the framework and the associated boilerplate. Furthermore, the
abscissa
CLI app ended up being a full-blown Abscissa application, allowing us to “eat our own dogfood” and use the acceptance testing functionality we developed as part of the framework in order to test the generator and ensure that generated applications both compile and pass their generated test suites (and additional tests included with the framework). - clap and structopt are widely recognized as the go-to solution for command line argument parsing, yet we are using the much more obscure gumdrop for this instead. Why not clap/structopt? There are a few different reasons for this: at the time we made the decision
clap
had dozens of transitive dependencies, whereasgumdrop
depends only ongumdrop_derive
which depends onproc-macro2
,quote
, andsyn
, which are all pretty much required to write Rust proc macros. In the intervening timeclap
/structopt
have reduced their number of dependencies, to the point that’s less of a concern. There are a few lingering problems though: wheregumdrop
andgumdrop_derive
are two crates in the same logical library which was written with a “proc macros-first” approach,clap
predates the widespread use of proc macros, andstructopt
feels to us at least like it’s been added as an afterthought. That may change though: the roadmap forclap
v3 both highlights these sorts of concerns and aims to address them. Our lingering concern is thatclap
duplicates a lot of functionality provided by Abscissa, hence its many dependencies, whereasgumdrop
has practically no dependencies. Wheregumdrop
lacks functionality, Abscissa already has that functionality and can leveragegumdrop
purely as a library while handling user-facing interactions itself. All of that said, we will be keeping our eye onclap
and may consider switching to it in the future.
Conclusion: what about the web? #
Throughout this post we talked about how Abscissa is useful for network services, and also highlighted the Dropwizard and Ruby on Rails, both web frameworks, were two of Abscissa’s biggest inspirations. Yet right now Abscissa has no story around the web, and if it did, what does that mean for people who are developing CLI utilities with it who don’t want the framework junked up with a bunch of web service functionality they aren’t using?
To address the concerns of anyone worried about the latter: we promise we will not put any web-related functionality into abscissa_core
. Though its number of dependencies may grow above 30 (we hope not that much, though!), we promise that pulling abscissa_core
into your application will never pull in a web framework, or Tokio. We will try to keep it as slim as possible.
We absolutely want to build Abscissa applications that use Tokio and function as web (and/or gRPC) service(s). Rather than putting this functionality directly into abscissa_core
, we can extend Abscissa with components that e.g. run the Tokio reactor and provide web service functionality.
We do not plan on making Abscissa into a bespoke web framework, but will most likely select some existing mature crates to provide this functionality. In doing so we will pay particular attention to things like the amount of unsafe code they use, and as mentioned earlier this will figure into our selection more than things like performance or functionality. All of that said, we have our eye on projects like warp and tower as potential building blocks for Abscissa web functionality.
In building out web functionality within Abscissa, we hope to address some problems we currently feel are unsolved within the Rust ecosystem, namely providing a secure-by-default web experience for Rust application development where various web security features (CSP, CSRF and/or SameSite cookies, mandatory HTTPS, authentication, authorization, sessions, etc) come free. This is an area where we feel Ruby on Rails has done a good job, and we would like to make it just as easy to use an application boilerplate generator to get a new Rust web application which does not need a lot of additional work to make it secure.
We are excited to see async/await land in Rust 1.38, and can’t wait to start using it, in conjunction with Abscissa, to build secure Rust web applications.