3 ways you can improve your npm publish

#anchorLet CI do the work

While you can run npm publish locally, it can cause issues. I'm personally using JetBrains IntelliJ to develop software and it puts an .idea folder in each project. I've stopped counting how many times I've accidentally published this folder to npm in the past…

I've managed to not commit this folder to any of our git repositories because I've set up a global .gitignore file, but unfortunately no such thing exists for .npmignore files. 😢

There are obviously other ways to avoid this issue, but we've found that one of the easiest ways is to just not publish from your local machine, and instead have your CI workflow publish the package whenever you push a tag to the repository. As an added benefit: you'll never forget to push the tag to the repository because it's now required to actually publish the package! 🥳

How to set this up largely depends on what CI system you are using. For our open-source projects we use GitHub Actions these days, and a typical release workflow file looks like this:

name: Release

on:
  push:
    tags:
      - "*"

jobs:
  release:
    name: Release
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: 14.x
          registry-url: "https://registry.npmjs.org"

      - run: npm publish
        env:
          NODE_AUTH_TOKEN: $

Let's go through this file step by step…

The name field describes what this workflow will be called in the GitHub Actions user interface.

The on object specifies when this workflow should run. In this case we are configuring it to run whenever a tag is pushed to the repository, and the * wildcard means that we don't care about what the exact tag name is.

jobs is a bit misleading, because in this workflow we only have a single job: release. In this job we will first copy the code onto the CI machine (checkout), and then setup Node.js. It is important here to explicitly specify the registry-url because otherwise the following npm publish command won't work. Here, we just use the official npm registry, but it is equally possible to publish packages to a company-internal private registry, if needed.

Lastly, we call npm publish. For this to succeed we need to be logged in… which we are not 🙈

An alternative to using npm login on CI is providing an "access token" through e.g. the NODE_AUTH_TOKEN environment variable. But where do you get such a token? The answer is: on the npm website:

• Go to https://www.npmjs.com
• Log in, if you haven't already
• Click your avatar on the top right corner of the page
• Click on "Access Tokens"
• Click the "Generate New Token" button
• Select the "Automation" token type
• Click the "Generate Token" button

Now you can copy the newly generated token and save it as a NPM_TOKEN secret in the settings of your GitHub repository.

While this generally works great for us, there are currently at least two downsides of this approach that you should be aware of:

• There is no difference anymore between commit/push access and publish access. This means that anyone that can push to your repository can now also publish new releases on npm. This is usually not a big problem, but you should be aware that it removes one of the security layers from the publishing process in favor of the CI publishing convenience.

• The generated npm token currently provides publish access to all npm packages that the account has access to. Fixing this is on the roadmap of the npm team, and at some point you will be able to generate tokens that can only access a subset of your projects.

For us, the convenience of only having to push a tag currently outweighs the disadvantages mentioned above, but it is still valuable to know what the tradeoffs are.

#anchorKeep a change log

When you see that one of your dependencies has an update available, how do you figure out what changed between your current version and the update?

You can look at the diff of the two versions, but that usually requires somewhat deep knowledge of how the dependency works internally. In an ideal world every project would have a CHANGELOG.md file, that describes what has changed between the individual versions. Unfortunately, not all projects keep such a file because maintaining such a file is often perceived as a significant workload.

To reduce the work that is needed to maintain a changelog file we can use changelog generators. These tools usually look at the git commits between the current version and the last release and output an overview of what the relevant changes were.

The changelog generator that we like to use at Mainmatter is: lerna-changelog.

We don't want to go into too much detail here, but this is essentially how it works: lerna-changelog looks at the commits and searches for commit messages that look like pull-request merge commits. It then requests more information about the corresponding pull-requests from the GitHub API, including the labels of that pull-request. Next, it groups the PRs by their labels and outputs them based on that grouping.

Here is an example of the lerna-changelog changelog itself:

## v2.0.1 (2021-08-07)

#### :bug: Bug Fix

- [#296](https://github.com/lerna/lerna-changelog/pull/296) Omit commiters line
  when all are filtered out ([@petrch87](https://github.com/petrch87))
- [#398](https://github.com/lerna/lerna-changelog/pull/398) Fix handling of
  --next-version-from-metadata option
  ([@contolini](https://github.com/contolini))

#### :house: Internal

- [#494](https://github.com/lerna/lerna-changelog/pull/494) Update yargs to
  v17.x ([@Turbo87](https://github.com/Turbo87))
- [#493](https://github.com/lerna/lerna-changelog/pull/493) CI: Run
  `pnpm install` before `npm publish` ([@Turbo87](https://github.com/Turbo87))

#### Committers: 3

- Chris Contolini ([@contolini](https://github.com/contolini))
- Petr Chňoupek ([@petrch87](https://github.com/petrch87))
- Tobias Bieniek ([@Turbo87](https://github.com/Turbo87))

One thing to be aware of is that lerna-changelog will only include PRs in the changelog that have one of these supported labels:

• breaking (💥 Breaking Change)
• enhancement (🚀 Enhancement)
• bug (🐛 Bug Fix)
• documentation (📝 Documentation)
• internal (🏠 Internal)

These labels are configurable, but using a label that is not configured might cause the PR to not show up in the listing. This behavior is an ongoing discussion and might change in the future, but this is currently how it works.

The way to use lerna-changelog is: when you bump the version in your package.json file, commit that change, and tag the commit, but you should also update the CHANGELOG.md with the latest changes for that particular version:

npx lerna-changelog

lerna-changelog usually figures out automatically what the last published version was, but if that does not work you can help by providing the --from CLI option. Similarly, if it can't figure out the GitHub URL of the project, you can help by setting it manually via --repo.

#anchorAutomate everything

Now that we've made our publishing process more complicated again by needing to update the changelog, let's simplify it by automating everything. 🤖

There is a CLI tool on npm called release-it, which describes itself as:

Generic CLI tool to automate versioning and package publishing related tasks

That sounds like exactly what we need! 😊

Our configuration file for release-it usually looks like this:

module.exports = {
  plugins: {
    'release-it-lerna-changelog': {
      infile: 'CHANGELOG.md',
    },
  },
  git: {
    commitMessage: 'v${version}',
    tagName: 'v${version}',
  },
  github: {
    release: true,
    releaseName: 'v${version}',
    tokenRef: 'GITHUB_AUTH',
  },
  npm: {
    publish: false,
  },
};

In the plugins section we specify that we want to use the release-it-lerna-changelog plugin to integrate lerna-changelog in our release process.

The git section configures what the commit message and tag names are supposed to look like, and in the github section we specify that we would also like to automatically create a "Release" on GitHub with the corresponding changelog attached.

Finally, we want to take advantage of our automatic CI publishing process, so we tell release-it to not run npm publish for us.

After adding release-it and release-it-lerna-changelog as dev dependencies, and putting the configuration above in a .release-it.js file we can run the tool to see how it works:

npx release-it

The first thing that release-it does is assembling the changelog, so that you can better judge whether this should be a major, minor or patch release. Conveniently, it asks you this exact question right after showing you the changelog preview.

Once you have chosen the version number, it will update the version field in the package.json file and update the CHANGELOG.md. Afterwards it will ask you to confirm that the files should be committed like this, and if you confirm, it will ask you whether the new commit should now be tagged. The cool part is that you can abort at any time and release-it will automatically clean up and revert you to the same state as before.

The final two confirmations are for pushing the commit and tag to GitHub and then creating the Release on GitHub. Once all of this is confirmed you can sit back and watch the CI machines take over.

To summarize, releasing a new version is now only a matter of running release-it, choosing a version number and then confirming a few actions, which usually shouldn't take more than a few seconds. As a bonus, this updated release process will automatically maintain the CHANGELOG.md file for you!

The instructions in this blog post were primarily aimed at JavaScript projects that are hosted on GitHub, but we've also set up similar processes, tools and automations for projects on private GitLab instances, Rust projects, and also with different changelog generators. If you need any help setting this up, don't hesitate to contact us.