@mapbox/node-pre-gyp

@mapbox/node-pre-gyp makes it easy to publish and install Node.js C++ addons from binaries

@mapbox/node-pre-gyp stands between npm and node-gyp and offers a cross-platform method of binary deployment.

Special note on previous package

On Feb 9th, 2021 @mapbox/node-pre-gyp@1.0.0 was released. Older, unscoped versions that are not part of the @mapbox org are deprecated and only @mapbox/node-pre-gyp will see updates going forward. To upgrade to the new package do:

npm uninstall node-pre-gyp --save
npm install @mapbox/node-pre-gyp --save

Features

• A command line tool called node-pre-gyp that can install your package’s C++ module from a binary.
• A variety of developer targeted commands for packaging, testing, and publishing binaries.
• A JavaScript module that can dynamically require your installed binary: require('@mapbox/node-pre-gyp').find

For a hello world example of a module packaged with node-pre-gyp see https://github.com/springmeyer/node-addon-example and the wiki for real world examples.

Credits

• The module is modeled after node-gyp by @Tootallnate
• Motivation for initial development came from @ErisDS and the Ghost Project.
• Development is sponsored by Mapbox

FAQ

See the Frequently Ask Questions.

Depends

• Node.js >= node v8.x

Install

node-pre-gyp is designed to be installed as a local dependency of your Node.js C++ addon and accessed like:

./node_modules/.bin/node-pre-gyp --help

But you can also install it globally:

npm install @mapbox/node-pre-gyp -g

Usage

Commands

View all possible commands:

node-pre-gyp --help

• clean – Remove the entire folder containing the compiled .node module
• install – Install pre-built binary for module
• reinstall – Run “clean” and “install” at once
• build – Compile the module by dispatching to node-gyp or nw-gyp
• rebuild – Run “clean” and “build” at once
• package – Pack binary into tarball
• testpackage – Test that the staged package is valid
• publish – Publish pre-built binary
• unpublish – Unpublish pre-built binary
• info – Fetch info on published binaries

You can also chain commands:

node-pre-gyp clean build unpublish publish info

Options

Options include:

• -C/--directory: run the command in this directory
• --build-from-source: build from source instead of using pre-built binary
• --update-binary: reinstall by replacing previously installed local binary with remote binary
• --runtime=node-webkit: customize the runtime: node, electron and node-webkit are the valid options
• --fallback-to-build: fallback to building from source if pre-built binary is not available
• --target=0.4.0: Pass the target node or node-webkit version to compile against
• --target_arch=ia32: Pass the target arch and override the host arch. Any value that is supported by Node.js is valid.
• --target_platform=win32: Pass the target platform and override the host platform. Valid values are linux, darwin, win32, sunos, freebsd, openbsd, and aix.

Both --build-from-source and --fallback-to-build can be passed alone or they can provide values. You can pass --fallback-to-build=false to override the option as declared in package.json. In addition to being able to pass --build-from-source you can also pass --build-from-source=myapp where myapp is the name of your module.
For example: npm install --build-from-source=myapp. This is useful if:

• myapp is referenced in the package.json of a larger app and therefore myapp is being installed as a dependency with npm install.
• The larger app also depends on other modules installed with node-pre-gyp
• You only want to trigger a source compile for myapp and the other modules.

Configuring

This is a guide to configuring your module to use node-pre-gyp.

1) Add new entries to your package.json

• Add @mapbox/node-pre-gyp to dependencies
• Add aws-sdk as a devDependency
• Add a custom install script
• Declare a binary object

This looks like:

"dependencies"  : {
      "@mapbox/node-pre-gyp": "1.x"
    },
    "devDependencies": {
      "aws-sdk": "2.x"
    }
    "scripts": {
        "install": "node-pre-gyp install --fallback-to-build"
    },
    "binary": {
        "module_name": "your_module",
        "module_path": "./lib/binding/",
        "host": "https://your_module.s3-us-west-1.amazonaws.com"
    }

For a full example see node-addon-examples’s package.json.
Let’s break this down:

• Dependencies need to list node-pre-gyp
• Your devDependencies should list aws-sdk so that you can run node-pre-gyp publish locally or a CI system. We recommend using devDependencies only since aws-sdk is large and not needed for node-pre-gyp install since it only uses http to fetch binaries
• Your scripts section should override the install target with "install": "node-pre-gyp install --fallback-to-build". This allows node-pre-gyp to be used instead of the default npm behavior of always source compiling with node-gyp directly.
• Your package.json should contain a binary section describing key properties you provide to allow node-pre-gyp to package optimally. They are detailed below.

Note: in the past we recommended putting @mapbox/node-pre-gyp in the bundledDependencies, but we no longer recommend this. In the past there were npm bugs (with node versions 0.10.x) that could lead to node-pre-gyp not being available at the right time during install (unless we bundled). This should no longer be the case. Also, for a time we recommended using "preinstall": "npm install @mapbox/node-pre-gyp" as an alternative method to avoid needing to bundle. But this did not behave predictably across all npm versions – see #260 for the details. So we do not recommend using preinstall to install @mapbox/node-pre-gyp. More history on this at fsevents/fsevents#157 (comment).

The binary object has three required properties

module_name

The name of your native node module. This value must:

• Match the name passed to the NODE_MODULE macro
• Must be a valid C variable name (e.g. it cannot contain -)
• Should not include the .node extension.

module_path

The location your native module is placed after a build. This should be an empty directory without other Javascript files. This entire directory will be packaged in the binary tarball. When installing from a remote package this directory will be overwritten with the contents of the tarball.
Note: This property supports variables based on Versioning.

host

A url to the remote location where you’ve published tarball binaries (must be https not http).
It is highly recommended that you use Amazon S3. The reasons are:

• Various node-pre-gyp commands like publish and info only work with an S3 host.
• S3 is a very solid hosting platform for distributing large files.
• We provide detail documentation for using S3 hosting with node-pre-gyp.

Why then not require S3? Because while some applications using node-pre-gyp need to distribute binaries as large as 20-30 MB, others might have very small binaries and might wish to store them in a GitHub repo. This is not recommended, but if an author really wants to host in a non-S3 location then it should be possible.
It should also be mentioned that there is an optional and entirely separate npm module called node-pre-gyp-github which is intended to complement node-pre-gyp and be installed along with it. It provides the ability to store and publish your binaries within your repositories GitHub Releases if you would rather not use S3 directly. Installation and usage instructions can be found here, but the basic premise is that instead of using the node-pre-gyp publish command you would use node-pre-gyp-github publish.

The binary object other optional S3 properties

If you are not using a standard s3 path like bucket_name.s3(.-)region.amazonaws.com, you might get an error on publish because node-pre-gyp extracts the region and bucket from the host url. For example, you may have an on-premises s3-compatible storage server, or may have configured a specific dns redirecting to an s3 endpoint. In these cases, you can explicitly set the region and bucket properties to tell node-pre-gyp to use these values instead of guessing from the host property. The following values can be used in the binary section:

host

The url to the remote server root location (must be https not http).

bucket

The bucket name where your tarball binaries should be located.

region

Your S3 server region.

s3ForcePathStyle

Set s3ForcePathStyle to true if the endpoint url should not be prefixed with the bucket name. If false (default), the server endpoint would be constructed as bucket_name.your_server.com.

The binary object has optional properties

remote_path

It is recommended that you customize this property. This is an extra path to use for publishing and finding remote tarballs. The default value for remote_path is "" meaning that if you do not provide it then all packages will be published at the base of the host. It is recommended to provide a value like ./{name}/v{version} to help organize remote packages in the case that you choose to publish multiple node addons to the same host.
Note: This property supports variables based on Versioning.

package_name

It is not recommended to override this property unless you are also overriding the remote_path. This is the versioned name of the remote tarball containing the binary .node module and any supporting files you’ve placed inside the module_path directory. Unless you specify package_name in your package.json then it defaults to {module_name}-v{version}-{node_abi}-{platform}-{arch}.tar.gz which allows your binary to work across node versions, platforms, and architectures. If you are using remote_path that is also versioned by ./{module_name}/v{version} then you could remove these variables from the package_name and just use: {node_abi}-{platform}-{arch}.tar.gz. Then your remote tarball will be looked up at, for example, https://example.com/your-module/v0.1.0/node-v11-linux-x64.tar.gz.
Avoiding the version of your module in the package_name and instead only embedding in a directory name can be useful when…