I have a collection of snippets and utilities that I frequently reach for when building web stuff. Up until now this code has been managed in a very adhoc fashion – copied and pasted between codebases, un-versioned, and free from the burden of tests 😉
The temptation is to publish these utilities, collectively or individually, on a package registry such as NPM. But, as rewarding and exhilarating as it can be to open source code, it does have its downsides. In particular, publicly publishing a package can signal to other developers that it's production-ready and bring with it the apparent obligation of supporting its use. Alternatively, sometimes the code is sensitive in nature or is not yet mature enough to see the light of day.
Publishing these packages privately is a good solution so long as it's economical and has an efficient enough workflow. To keep the organisational overhead low I'll keep them all in a single repository, following the monolithic repository pattern. (I can't help but feel "minilithic" would be a more appropriate name here.)
NPM doesn't allow users to publish private packages for free, but the GitHub Package Registry does (with strings attached). Given GitHub's recent acquisition of NPM this might well change in the future 🤷♂️
Setup the Mono-Repository
I'll use my nuxt-modules private GitHub repository, and the private packages
within, as a working example.
Let's get started... In a terminal of your choice create a new project directory and initialise Git and Yarn:
> mkdir nuxt-modules
> cd nuxt-modules
> git init
> yarn init
Enable Yarn Workspaces by
configuring the "workspaces" property in package.json:
{
"name": "nuxt-modules",
"private": true,
"workspaces": ["packages/*"]
}
Initialise Lerna with independent versioning enabled:
> lerna init --independent
Configure Lerna to play-nice with Yarn Workspaces and target the GitHub Package
Registry in lerna.json:
{
"packages": ["packages/*"],
"version": "independent",
"npmClient": "yarn",
"useWorkspaces": true,
"command": {
"publish": {
"conventionalCommits": true,
"message": "chore(release): publish",
"registry": "https://npm.pkg.github.com",
"allowBranch": "master"
}
}
}
Feel free to customise the other properties, these are just my preferences.
Create the Packages
Populate the packages/ directory with a sub-directory for each package. The
directory names shouldn't be prefixed with the scope, but the name field in
the package.json should, e.g. packages/nuxt-html-validate will contain a
package.json with the name field set to @saulhardman/nuxt-html-validate.
You can create packages using Lerna's lerna create command or by hand. The
bare-minimum for an NPM package is a JavaScript entry-point (e.g. index.js)
and a package.json.
Development dependencies that are common to all of the packages should be
installed in the mono-repository root. As an example, here's the command to
install ESLint, passing the -W argument to the add
command:
> yarn add --dev -W eslint
A critical step in this process is to run yarn init within each of the
directories. It's then necessary to make a minor adjustment to the resulting
package.json files to set the repository.directory and
publishConfig.registry fields. Here is an example of the
@saulhardman/nuxt-html-validate package which is located in the
packages/nuxt-html-validate/ sub-directory:
{
"repository": {
"type": "git",
"url": "ssh://git@github.com/saulhardman/nuxt-modules.git",
"directory": "packages/nuxt-html-validate"
},
"publishConfig": {
"registry": "https://npm.pkg.github.com/"
}
}
The final result should look something like this:
.
├── .gitignore
├── LICENSE.md
├── lerna.json
├── package.json
├── packages
│ ├── nuxt-html-validate
│ │ ├── README.md
│ │ ├── index.js
│ │ └── package.json
│ ├── nuxt-release
│ │ ├── README.md
│ │ ├── index.js
│ │ └── package.json
│ ├── nuxt-robotize
│ │ ├── README.md
│ │ ├── index.js
│ │ └── package.json
│ └── nuxt-rss
│ ├── README.md
│ ├── index.js
│ └── package.json
└── yarn.lock
Authenticate with the GitHub Package Registry
The next step is to authenticate with the Github Package Registry (replace
@saulhardman with your GitHub username):
> npm login --registry=https://npm.pkg.github.com --scope=@saulhardman
To interact with the package repository API, GitHub requires you to create a Personal Access Token (PAT) which you will use in-lieu of your password. Make sure that the 'repo', 'write:packages', 'read:packages', and 'delete:packages' options are selected:
With that in-hand the .npmrc is configured to point requests for
@saulhardman-scoped packages to GitHub (rather than NPM) and provide the PAT
as an authToken (replace TOKEN and @saulhardman with your respective
values):
//npm.pkg.github.com/:_authToken=TOKEN
@saulhardman:registry=https://npm.pkg.github.com
Even though this Git repository will be private it's good practice not to commit
keys and tokens. Accordingly, be sure to amend the .gitignore config to
include the .npmrc.
Publish the Packages
Create your private GitHub repository and push your initial commit containing
your packages. It's my preference to set the package.version fields to 0.0.0
to begin with. At publish-time you can pass minor or major to have 0.1.0
or 1.0.0 be the initial release version:
> yarn lerna publish minor # initial release 0.1.0
> yarn lerna publish major # initial release 1.0.0
Once you've received a "Package published" response, you will be able to view your packages on the GitHub repository page:
Installing Private GitHub Packages
The permissions workflow surrounding private packages is... not great. There is, as far as I'm aware, no way to scope PATs to organisations, repositories, or packages. The method outlined here will allow you to install all private packages that your GitHub account has access to.
To install a private package all that's required is an .npmrc to assign an
access token and configure the scopes. The PAT could be the same one used above
or a different PAT with read-only permissions (replace TOKEN with your PAT and
@saulhardman with your GitHub username):
//npm.pkg.github.com/:_authToken=TOKEN
@saulhardman:registry=https://npm.pkg.github.com
Only packages in the scope @saulhardman will be installed from the GitHub
Package Registry – all others will default to NPM. The yarn add command can be
used as usual, e.g.:
> yarn add @saulhardman/nuxt-html-validate
Installing Private GitHub Packages from GitHub Actions
Setting the NODE_AUTH_TOKEN environment variable on the yarn install step
should be enough, but in my experience it is not. There is a thread on the
GitHub Community Forum
documenting a number of people's struggles.
An alternative – whether you're running yarn install directly or using a
third-party action such as
bahmutov/npm-install – is to
construct an .npmrc dynamically using a PAT stored as an
encrypted secret:
steps:
- name: Configure NPM
run: |
echo "//npm.pkg.github.com/:_authToken=$NODE_AUTH_TOKEN" > .npmrc
echo '@saulhardman:registry=https://npm.pkg.github.com' >> .npmrc
env:
NODE_AUTH_TOKEN: ${{ secrets.NODE_AUTH_TOKEN }}
- name: Install Yarn Dependencies
uses: bahmutov/npm-install@v1
Closing Thoughts
I've created a number of private packages over the last few months – ranging from the Nuxt modules outlined above to Vue components and JavaScript utilities. I've thoroughly enjoyed it so far and I feel the initial overhead will be well worth the reward in the long term.
Discovering a bug in one usage context, fixing it, adding a test case if necessary, and then having that update trickle-down to other projects with very little friction is both satisfying and refreshing.
