Flecks of Rust #10: Designing a Rust crate

Rust has a fairly small standard library, and the Rust ecosystem relies a lot on crates, or small Rust libraries that can be integrated into your progam.

While it is easy to use crates in your program with Cargo, it is also very easy to make your own crates and even publish them on crates.io, the Rust crate repository.

The process of making a new crate consists of using Cargo to create a new library, writing the code for any data types and subprograms, testing them, and finally publishing the crate on crates.io.

In this installment I will show you how to make a new crate called nybble. It contains code to split a vector of bytes (a Vec<u8>) into pairs of half-bytes, 4-bit entities known as a nybble or nibble. (The more common spelling is nibble, but nybble goes well with byte, don't you think?)

For example, if you have the byte 0x42, the result of splitting that into nybbles will be 0x04 and 0x02. But which order should the nybbles be? You need to make a choice, even though the most common way seems to be high nybble (the most significant half) first.

Create a crate with Cargo

The first thing to do is to open a terminal and change to the directory where you keep your Rust projects, and issue the command to create a new library:

% cargo new --lib nybble
    Created library `nybble` package

Cargo reports back that it has created a new library. The resulting files will be standard fare: a Cargo.toml file and src/lib.rs. Next you will be editing these.

In this crate, the library source code in src/lib.rs looks like this:

//! # nybble
//!
//! `nybble` is a helper crate to split byte vectors into nybbles
//! and combine them back.

/// The order of nybbles in a byte.
#[derive(Copy, Clone, Eq, PartialEq, Hash, Debug)]
pub enum NybbleOrder {
    HighFirst,
    LowFirst
}

/// Gets the high nybble from a byte.
pub fn high_nybble(b: u8) -> u8 {
    (b & 0xf0) >> 4
}

/// Gets the low nybble from a byte.
pub fn low_nybble(b: u8) -> u8 {
    b & 0x0f
}

/// Gets the high and low nybble of a byte as a tuple,
/// with the high nybble first.
pub fn nybbles_from_byte(b: u8) -> (u8, u8) {
    (high_nybble(b), low_nybble(b))
}

/// Makes a byte from the high and low nybbles,
/// with the high nybble specified first.
pub fn byte_from_nybbles(high: u8, low: u8) -> u8 {
    high << 4 | low
}

/// Make a new byte array from `data` with the bytes split into
/// high and low nybbles. The `order` argument determines
/// which one comes first.
pub fn nybblify(data: Vec<u8>, order: NybbleOrder) -> Vec<u8> {
    let mut result = Vec::<u8>::new();

    for b in data {
        let n = nybbles_from_byte(b);
        if order == NybbleOrder::HighFirst {
            result.push(n.0);
            result.push(n.1);
        } else {
            result.push(n.1);
            result.push(n.0);
        }
    }

    result
}

/// Make a new byte array from `data` by combining adjacent bytes
/// representing the high and low nybbles of each byte.
/// The `order` argument determines which one comes first.
pub fn denybblify(data: Vec<u8>, order: NybbleOrder) -> Vec<u8> {
    assert_eq!(data.len() % 2, 0);  // length must be even

    let mut result = Vec::<u8>::new();

    let mut index = 0;
    let mut offset = 0;
    let count = data.len() / 2;

    while index < count {
        let high = data[offset];
        let low = data[offset + 1];
        let b = if order == NybbleOrder::HighFirst {
            byte_from_nybbles(high, low)
        } else {
            byte_from_nybbles(low, high)
        };
        result.push(b);
        index += 1;
        offset += 2;
    }

    result
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_nybblify() {
        let b = vec![0x01, 0x23, 0x45];
        let nb = nybblify(b, NybbleOrder::HighFirst);
        assert_eq!(nb, vec![0x00, 0x01, 0x02, 0x03, 0x04, 0x05]);
    }

    #[test]
    fn test_nybblify_flipped() {
        let b = vec![0x57, 0x61, 0x76];
        let nb = nybblify(b, NybbleOrder::LowFirst);
        assert_eq!(nb, vec![0x07, 0x05, 0x01, 0x06, 0x06, 0x07]);
    }

    #[test]
    fn test_denybblify() {
        let b = vec![0x00, 0x01, 0x02, 0x03, 0x04, 0x05];
        let nb = denybblify(b, NybbleOrder::HighFirst);
        assert_eq!(nb, vec![0x01, 0x23, 0x45]);
    }

    #[test]
    fn test_denybblify_flipped() {
        let b = vec![0x07, 0x05, 0x01, 0x06, 0x06, 0x07];
        let nb = denybblify(b, NybbleOrder::LowFirst);
        assert_eq!(nb, vec![0x57, 0x61, 0x76]);
    }
}

Building and testing

A library crate is built just like a normal program:

% cargo build
    Compiling nybble v0.1.0 (/Users/me/Projects/Rust/nybble)
     Finished dev [unoptimized + debuginfo] target(s) in 0.23s

You can also build a release version:

% cargo build --release
    Compiling nybble v0.1.0 (/Users/me/Projects/Rust/nybble)
     Finished release [optimized] target(s) in 0.56s

The library includes some tests, and you can run them with Cargo:

% cargo test
    Compiling nybble v0.1.0 (/Users/me/Projects/Rust/nybble)
     Finished test [unoptimized + debuginfo] target(s) in 0.40s
      Running unittests src/lib.rs (target/debug/deps/nybble-dca14a5d70abcbdd)

 running 4 tests
 test tests::test_denybblify_flipped ... ok
 test tests::test_denybblify ... ok
 test tests::test_nybblify ... ok
 test tests::test_nybblify_flipped ... ok

 test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

    Doc-tests nybble

 running 0 tests

 test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Preparing to publish the crate

Cargo has a command to publish the crate on crates.io. However, if you try it without committing the library to version control, you will not succeed:

% cargo publish
    Updating crates.io index
warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
error: 2 files in the working directory contain changes that were not yet committed into git:

Cargo.toml
src/lib.rs

to proceed despite this and include the uncommitted changes, pass the `--allow-dirty` flag

Also, you need to specify some important metadata about the crate. All of the information mentioned by Cargo goes into the Cargo.toml file. This is the initial version:

[package]
name = "nybble"
version = "0.1.0"
edition = "2021"
authors = ["Jere K├Ąpyaho <email@redacted>"]
license = "MIT"
description = """
Helper crate to split byte vectors into nybbles and combine them back.
"""
repository = "https://github.com/coniferprod/nybble"

[dependencies]

It is also a good idea to add a license file with the correct license text (in this case, the MIT License) and a `README.md` which currently just repeats the information in the top-level comment of the library.

When all the files have been created and updated, commit the changes to your local repository which was created by Cargo:

% git status
On branch main

No commits yet

Untracked files:
    (use "git add ..." to include in what will be committed)
    .gitignore
    Cargo.toml
    LICENSE
    README.md
    src/

nothing added to commit but untracked files present (use "git add" to track)
% git add .
% git commit -m "Initial commit"
[main (root-commit) 581e77a] Initial commit
    5 files changed, 152 insertions(+)
    create mode 100644 .gitignore
    create mode 100644 Cargo.toml
    create mode 100644 LICENSE
    create mode 100644 README.md
    create mode 100644 src/lib.rs

Create a public repository on GitHub, set the remote on your local repository, and push the changes:

% git remote add origin git@github.com:coniferprod/nybble.git
% git branch -M main
% git push -u origin main
Enumerating objects: 8, done.
Counting objects: 100% (8/8), done.
Delta compression using up to 6 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (8/8), 2.25 KiB | 2.25 MiB/s, done.
Total 8 (delta 0), reused 0 (delta 0), pack-reused 0
To github.com:coniferprod/nybble.git
    * [new branch]      main -> main
branch 'main' set up to track 'origin/main'.

If you want to publish your crate, you need to create an account on crates.io. The process is detailed in the Cargo book, so I won't repeat it here. Please be careful with your login token!

Before actually publishing, you should to a "dry run" to see if there is anything missing:

% cargo publish --dry-run
Updating crates.io index
Packaging nybble v0.1.0 (/Users/me/Projects/Rust/nybble)
Verifying nybble v0.1.0 (/Users/me/Projects/Rust/nybble)
Compiling nybble v0.1.0 (/Users/me/Projects/Rust/nybble/target/package/nybble-0.1.0)
Finished dev [unoptimized + debuginfo] target(s) in 1.13s
Packaged 7 files, 5.2KiB (2.3KiB compressed)
Uploading nybble v0.1.0 (/Users/me/Projects/Rust/nybble)
warning: aborting upload due to dry run

Note the last line of output: everything was fine, but the crate was not yet uploaded.

When you're absolutely sure that your crate is ready for the world, publish it with `cargo publish`. You should see additional lines of output like this:

Uploaded nybble v0.1.0 to registry `crates-io`
note: Waiting for `nybble v0.1.0` to be available at registry `crates-io`.
You may press ctrl-c to skip waiting; the crate should be available shortly.
    Published nybble v0.1.0 at registry `crates-io`

Now the crate has been sent on its way to crates.io and should soon appear in the catalog.

Viewing your crate

After a while your crate should be available on crates.io. At this point the `nybble` crate is on crates.io, ready to include in your program's dependencies in Cargo.toml.

Since the crate's source code has documentation comments, crates.io has also automatically generated online documentation.

In a future installment I will cover how to add binaries to the crate, so that you can include useful tools that exercise the functionality of the crate.