s452108/atcheck
2
1
Fork 0
Code Issues Pull Requests Releases Wiki Activity
ea92d2b906
atcheck/lsscanner/new_ubuntu/node_modules_win/npm/changelogs/CHANGELOG-3.md

247 KiB
Executable File
Raw Blame History

v3.10.10 (2016-11-04)

See the discussion on #14042 for more context on this release, which is intended to address a serious regression in shrinkwrap behavior in the version of the CLI currently bundled with Node.js 6 LTS "Boron". You should never install this version directly; instead update to npm@4, which has everything in this release and more.

REGRESSION FIX

  • 9aebe98 #14117 Fixes a bug where installing a shrinkwrapped package would fail if the platform failed to install an optional dependency included in the shrinkwrap. (@watilde)

UPDATE SUPPORT MATRIX

With the advent of the second official Node.js LTS release, Node 6.x 'Boron', the Node.js project has now officially dropped versions 0.10 and 0.12 out of the maintenance phase of LTS. (Also, Node 5 was never part of LTS, and will see no further support now that Node 7 has been released.) As a small team with limited resources, the npm CLI team is following suit and dropping those versions of Node from its CI test matrix.

v3.10.9 (2016-10-06)

Hi everyone! This is the last of our monthly releases. We're going to give an every-two-weeks schedule a try starting with our next release. We'll reevaluate in a quarter, but we suspect that will be what we'll stick with. You might be wondering why we've been fiddling with the release cadence? Well, we've been trying to tune it to to minimize the overhead for our little team.

This is ALSO the ULTIMATE release of npm version 3. That's right, in just two weeks' time (October 20th for you fans of calendar time), our dear npm will be hitting the big 4.0.

DON'T PANIC

This is gonna be a much, MUCH smaller major version than 3.x was. Maybe even smaller than 2.x was. I can't tell you everything that'll be in there just yet, but at the very least it's going to have what's in our 4.x milestone, PLUS, the first steps in making prepublish work the way people expect it to.

NOW ABOUT THIS RELEASE

This release sees a whole slew of bug fixes. Notably a bunch of lifecycle fixes and a really important shrinkwrap fix.

LIFECYCLE FIXES

  • d388f90 #13942 Fix current working directory while running shrinkwrap lifecycle scripts. Previously if you ran a shrinkwrap from another lifecycle script AND node_modules existed (and if you're running npm shrinkwrap it probably should) then npm would run the shrinkwrap lifecycle from the node_modules folder instead of the package folder. (@evocateur) (@iarna)
  • c3b6cdf #13964 Fix bug where the uninstall lifecycles weren't being run when you reinstalled/updated an existing module. (@iarna)
  • 72bb89c #13344 When running lifecycles use TMPDIR if it's writable and fall back to the current working directory if not. Previously we just assumed TMPDIR wouldn't be writable (as we might have been running as nobody and nobody on some systems can't write to TMPDIR). (@aaronjensen)

SHRINKWRAP GIT & TAGGED DEPENDENCY FIX

  • 3b5eee0 #13941 Fix git and tagged dependency matching with shrinkwraps. Previously git and tag (ie foo@latest) dependencies installed from a shrinkwrap would always be flagged as invalid. (@iarna)

BUG FIXES

  • bf3bd1e #14143 Fix bug in npm version where npm-shrinkwrap.json wouldn't be updated if you ran npm version from outside of your project root. (@lholmquist)
  • 1089878 #13613 Log 'skipping action' as 'verbose' instead of 'warn'. This removes a lot of clutter when there are links in your node_modules. The long term plan is to entirely blind npm to what's inside links, which will make this code go away entirely. (@timoxley)
  • 952f1e1 #13999 Fix a bug where setting bin to null in your package.json would result in npm crashing. (@IonicaBizau)
  • fcf8b11 #14032 When using npm view, if you specified a version that didn't exist it would previously print undefined (even if you asked for JSON output). It now prints nothing in this situation. This brings npm@3's behavior in line with npm@2. (@roblg)
  • 93c689f #14032 When using npm view --json with a version range that matches multiple versions we now return a list of all of the metadata for all of those versions. Previously we picked one and only returned that. This brings npm@3's behavior in line with npm@2. (@roblg)
  • 2411728 #14045 Fix a Windows-only bug in the git tests. The tests had rather particular ideas about what arguments would be passed to git and on Windows they got this wrong. (@watilde)

DOCUMENTATION & MISC

DEPENDENCY UPDATES

v3.10.8 (2016-09-08)

Monthly releases are so big! Just look at all this stuff!

Our quarter of monthly releases is almost over. The next one, in October, might very well be our last one as we move to trying something different and learning lessons from our little experiment.

You may also want to keep an eye our for npm@4 next month, since we're planning on finally releasing it then and including a (small) number of breaking changes we've been meaning to do for a long time. Don't worry, though: npm@3 will still be around for a bit and will keep getting better and better, and is most likely going to be the version that node@6 uses once it goes to LTS.

As some of us have mentioned before, npm is likely to start doing more regular semver-major bumps, while keeping those bumps significantly smaller than the huge effort that was npm@3 -- we're not very likely to do a world-shaking thing like that for a while, if ever.

All that said, let's move on to the patches included in v3.10.8!

SHRINKWRAP LEVEL UP

The most notable part of this release is a series of commits meant to make npm shrinkwrap more consistent. By itself, shrinkwrap seems like a fairly straightforward thing to implement, but things get complicated when it starts interacting with devDependencies, optionalDependencies, and bundledDependencies. These commits address some corner cases related to these.

  • a7eca32 #10073 Record if a dependency is only used as a devDependency and exclude it from the shrinkwrap file. (@bengl)
  • 1eabcd1 #10073 Record if a dependency is optional to shrinkwrap. (@bengl)
  • 03efc89 #13692 We were doing a weird thing where we used a package.json field installable to check to see if we'd checked for platform compatibility, and if not did so. But this was the only place that was ever done so there was no reason to implement it in such an obfuscated manner. Instead it now just directly checks and then records that its done so on the node object with knownInstallable. This is useful to know because modules expanded via shrinkwrap don't go through this– inflateShrinkwrap does not currently have any rollback semantics and so checking this sort of thing there is unhelpful. (@iarna)
  • ff87938 #11735 Running npm install --save-dev will now update shrinkwrap file, but only if there already are devDependencies in it. (@szimek)
  • c00ca3a #13394 Check installability of modules from shrinkwrap, since modules that came into the tree vie shrinkwrap won't already have this information recorded in advance. (@iarna)

INSTALLER ERROR REPORTING LEVEL UP

As part of the shrinkwrap push, there were also a lot of error-reporting improvements. Some to add more detail to error objects, others to fix bugs and inconsistencies.

MISCELLANEOUS BUGS LEVEL UP

  • 116b6c6 #13456 In lifecycle scripts, any node_modules/.bin existing in the hierarchy should be turned into an entry in the PATH environment variable. However, prior to this commit, it was splitting based on the string node_modules, rather than restricting it to only path portions like /node_modules/ or \node_modules\. So, a path containing an entry like my_node_modules would be improperly split. (@isaacs)
  • 0a28dd0 npm/fstream-npm#23 fstream-npm@1.2.0: Always ignore *.orig files, which are generated by git when using git mergetool, by default. (@zkat)
  • a3a2fb9 #13708 Always ignore *.orig files, which are generated by git when using git mergetool, by default. (@boneskull)

TOOLING LEVEL UP

DOCUMENTATION LEVEL UP

DEPENDENCY LEVEL UP

v3.10.7 (2016-08-11)

Hi all, today's our first release coming out of the new monthly release cadence. See below for details. We're all recovered from conferences now and raring to go! We've got some pretty keen bug fixes and a bunch of documentation and dependency updates. It's hard to narrow it down to just a few, but of note are scoped packages in bundled dependencies, the preinstall lifecycle fix, the shrinkwrap and Git dependencies fix and the fix to a crasher involving cycles in development dependencies.

NEW RELEASE CADENCE

Releasing npm has been, for the most part, a very prominent part of our weekly process process. As part of our efforts to find the most effective ways to allocate our team's resources, we decided last month that we would try and slow our releases down to a monthly cadence, and see if we found ourselves with as much extra time and attention as we expected to have. Process experiments are useful for finding more effective ways to do our work, and we're at least going to keep doing this for a whole quarter, and then measure how well it worked out. It's entirely likely that we'll switch back to a more frequent cadence, specially if we find that the value that weekly cadence was providing the community is not worth sacrificing for a bit of extra time. Does this affect you significantly? Let us know!

SCOPED PACKAGES IN BUNDLED DEPENDENCIES

Prior to this release and v2.15.10, npm had ignored scoped modules found in bundleDependencies.

preinstall LIFECYCLE IN CURRENT PROJECT

BETTER SHRINKWRAP WITH GIT DEPENDENCIES

  • 0f7e319 #12718 Update outdated git dependencies found in shrinkwraps. Previously, if the module version was the same then no update would be completed even if the committish had changed. (@kossnocorp)

CYCLES IN DEVELOPMENT DEPENDENCIES NO LONGER CRASH

  • 1691de6 #13327 Fix bug where cycles found in development dependencies could result in infinite recursion that resulted in crashes. (@iarna)

IMPROVE "NOT UPDATING LINKED MODULE" WARNINGS

  • 1619871 #12893 Only warn about symlink update if version number differs The update-linked action outputs a warning that it needs to update the linked package, but can't, There is no need for the package to be updated if it is already at the correct version. This change does a check before logging the warning. (@DaveEmmerson)

MORE BUG FIXES

  • 8f8d1b3 #11398 Fix bug where package.json files that contained a type property could cause crashes. type is not a package.json property that npm makes use of and having it should be (and now is) harmless. (@zkat)
  • e7fa6c6 #13353 Add GIT_EXEC_PATH to Git environment whitelist. (@mhart)
  • c23af21 #13626 Use HTTPS issues URL in the error message for type validation errors. (@watilde)

INCLUDE npm login IN COMMAND SUMMARY

  • ab0c4b1 #13581 The login command has long been an alias for adduser. At the same time, there is an expectation not just of that particular word being something to look for, but of there being clear symmetry with logout. So it was a bit confusing when login didn't show up in npm help on a technicality. This seems like an acceptable exception to the rule that says "no aliases in npm help". (@zkat)

DOCUMENTATION

DEPENDENCIES

  • 124427e #8614 fstream-npm@1.1.1: Fixes bug with inclusion of scoped bundled dependencies. (@forivall)

  • 7e0cdff #13497 graceful-fs@4.1.5: graceful-fs had a bug fix which fixes a problem (nodejs/node#7846) exposed by recent changes to Node.js. (@thefourtheye)

  • 9b88cb8 #9984 request@2.74.0: Update request library to at least 2.73 to fix a bug where npm install would crash with Cannot read property 'emit' of null.

    Update request dependency tough-cookie to 2.3.0 to to address https://nodesecurity.io/advisories/130. Versions 0.9.7 through 2.2.2 contain a vulnerable regular expression that, under certain conditions involving long strings of semicolons in the "Set-Cookie" header, causes the event loop to block for excessive amounts of time. (@zarenner) (@stash-sfdc)

  • bf78ce5 #13387 minimatch@3.0.3: Handle extremely long and terrible patterns more gracefully. There were some magic numbers that assumed that every extglob pattern starts and ends with a specific number of characters in the regular expression. Since !(||) patterns are a little bit more complicated, this led to creating an invalid regular expression and throwing. (@isaacs)

  • 803e538 isaacs/rimraf#111 rimraf@2.5.4: Clarify assertions: cb is required, options are not. (@isaacs)

  • a9f84ef lodash.without@4.2.0 (@jdalton)

  • f59ff1c lodash.uniq@4.4.0 (@jdalton)

  • 8cc027e lodash.union@4.5.0 (@jdalton)

  • 0a6c1e4 lodash.without@4.3.0 (@jdalton)

  • 4ab0181 lodash.clonedeep@4.4.1 (@jdalton)

v3.10.6 (2016-07-07)

This week we have a bunch of bug fixes for ya! A shrinkwrap regression introduced in 3.10.0, better lifecycle PATH behavior, improvements when working with registries other than registry.npmjs.org and a fix for hopefully the last don't print a progress bar over my interactive thingy bug.

SHRINKWRAP AND DEV DEPENDENCIES

The rewrite in 3.10.0 triggered a bug where dependencies of devDependencies would be included in your shrinkwrap even if you didn't request devDependencies.

  • 2484529 #13308 Fix bug where deps of devDependencies would be incorrectly included in shrinkwraps. (@iarna)

BETTER PATH LIFECYCLE BEHAVIOR

We've been around the details on this one a few times in recent months and hopefully this will bring is to where we want to be.

  • 81051a9 #12968 When running lifecycle scripts, only prepend directory containing the node binary to PATH if not already in PATH. (@segrey)

BETTER INTERACTIONS WITH THIRD PARTY REGISTRIES

  • 071193c #10869 If the registry returns a list of versions some of which are invalid, skip those when picking a version to install. This can't happen with registry.npmjs.org as it will normalize versions published with it, but it can happen with other registries. (@gregersrygg)

ONE LAST TOO-MUCH-PROGRESS CORNER

HTML DOCS IMPROVEMENTS

DEPENDENCY UPDATES (THAT MATTER)

v3.10.5 (2016-07-05)

This is a fix to this week's testing release to correct the update of node-gyp which somehow got mangled.

v3.10.4 (2016-06-30)

Hey y'all! This release includes a bunch of fixes we've been working on as we continue on our big-bug push. There's still a lot of it left to do, but once this is done, things should just generally be more stable, installs should be more reliable and correct, and we'll be able to move on to more future work. We'll keep doing our best! 🙌

RACES AS WACKY AS REDLINE

Races are notoriously hard to squash, and tend to be some of the more common recurring bugs we see on the CLI. @julianduque did some pretty awesome sleuthing work to track down a cache race and helpfully submitted a patch. There were some related races in the same area that also got fixed at around the same time, mostly affecting Windows users.

  • 2a37c97 #12669 #13023 The CLI is pretty aggressive about correcting permissions across the cache whenever it writes to it. This aggressiveness caused a couple of races where temporary cache files would get picked up by fs.readdir, and removed before chownr was called on them, causing ENOENT errors. While the solution might seem a bit hamfisted, it's actually perfectly safe and appropriate in this case to just ignore those resulting ENOENT errors. (@julianduque)
  • ea018b9 #13023 If a user were to have SUDO_UID and SUDO_GID, they'd be able to get into a pretty weird state. This fixes that corner case. (@zkat)
  • 703ca3a #13023 A missing return was causing chownr to be called on Windows, even though that's literally pointless, and causing crashes in the process, instead of short-circuiting. This was entirely dependent on which callback happened to be called first, and in some cases, the failing one would win the race. This should prevent this from happening in the future. (@zkat)
  • 69267f4 #13023 Added tests to verify correct-mkdir race patch. (@zkat)
  • e5f50ea #13023 Added tests to verify addLocal race patch. (@zkat)

SHRINKWRAP IS COMPLICATED BUT IT'S BETTER NOW

@iarna did some heroic hacking to refactor a bunch of shrinkwrap-related bits and fixed some resolution and pathing issues that were biting users. The code around that stuff got more readable/maintainable in the process, too!

  • 346bba1 #13214 Resolve local dependencies in npm-shrinkwrap.json relative to the top of the tree. (@iarna)
  • 4a67fdb #13213 If you run npm install modulename it should, if a npm-shrinkwrap.json is present, use the version found there. If not, it'll use the version found in your package.json, and failing that, use latest. This fixes a case where the first check was being bypassed because version resolution was being done prior to loading the shrinkwrap, and so checks to match the shrinkwrap version couldn't succeed. (@iarna)
  • afa2133 #13214 Refactor shrinkwrap specifier lookup into shared function. (@iarna)
  • 2820b56 #13214 Refactor operations in inflate-shrinkwrap.js into separate functions for added clarity. (@iarna)
  • ee5bfb3 Fix Windows path issue in a shrinkwrap test. (@zkat)

OTHER BUGFIXES

  • a11a7b2 #13212 Resolve local paths passed in through the command line relative to current directory, instead of relative to the package.json. (@iarna)

DEPENDENCY UPDATES

DOCUMENTATION FIXES

v3.10.3 (2016-06-23)

Given that we had not one, but two updates to our RC this past week, it should come as no surprise that this week's full release is a bit lighter. We have some documentation patches and a couple of bug fixes via dependency updates.

If you haven't yet checked out last week's release, v3.10.0 and the two follow up releases v3.10.1 and v3.10.2, you really should do so. They're the most important releases we've had in quite a while, fixing a bunch of critical bugs (including an issue impacting publishing with Node.js 6.x) and of course, bringing in the new and improved progress bar.

BUM SYMLINKS BURN NO MORE

There's been a bug lurking where broken symlinks in your node_modules folder could cause all manner of mischief, from crashes to empty npm ls results. The intrepid @watilde tracked this down for us.

This addresses the root cause of the outdated crasher we protected against earlier this week in #13115.

This also fixes #9564, the problem where a bad symlink in your global modules would result in an empty result when you ran npm ls -g.

This ALSO likely fixes numerous "Missing argument #1" errors. (But surely not all of them as that's actually just a generic arity and type-validation failure.)

BETTER UNICODE DETECTION

DOCUMENTATION FIXES

DEPENDENCY UPDATES

  • e44d2db aproba@1.0.4: Documentation updates and minor refactoring. (@iarna)

v3.10.2 (2016-06-17):

This is a quick hotfix release with two small bug fixes. First, there was an issue where the new progress bar would overwrite interactive prompts, that is, those found in npm login and npm init. Second, if the directory you were running npm outdated on was a bad link or otherwise had unrecoverable errors then npm would crash instead of printing the error.

v3.10.1 (2016-06-17):

There are two very important bug fixes and one long-awaited (and significant!) deprecation in this hotfix release. Hold on.

WHOA

When Node.js 6.0.0 was released, the CLI team noticed an alarming upsurge in bugs related to important files (like README.md) not being included in published packages. The new bugs looked much like #5082, which had been around in one form or another since April, 2014. #5082 used to be a very rare (and obnoxious) bug that the CLI team hadn't had much luck reproducing, and we'd basically marked it down as a race condition that arose on machines using slow and / or rotating-media-based hard drives.

Under 6.0.0, the behavior was reliable enough to be nearly deterministic, and made it very difficult for publishers using .npmignore files in combination with "files" stanzas in package.json to get their packages onto the registry without one or more files missing from the packed tarball. The entire saga is contained within the issue, but the summary is that an improvement to the performance of fs.realpath() made it much more likely that the packing code would lose the race.

Fixing this has proven to be very difficult, in part because the code used by npm to produce package tarballs is more complicated than, strictly speaking, it needs to be. @evanlucas contributed a patch that passed the tests in a special test suite that I (@othiym23) created (with help from @addaleax), but only after we'd released the fixed version of that package did we learn that it actually made the problem worse in other situations in npm proper. Eventually, @rvagg put together a more durable fix that appears to completely address the errant behavior under Node.js 6.0.0. That's the patch included in this release. Everybody should chip in for redback insurance for Rod and his family; he's done the community a huge favor.

Does this mean the long (2+ year) saga of #5082 is now over? At this point, I'm going to quote from my latest summary on the issue:

The CLI team (mostly me, with input from the rest of the team) has decided that the overall complexity of the interaction between fstream, fstream-ignore, fstream-npm, and node-tar has grown more convoluted than the team is comfortable (maybe even capable of) supporting.

  • While I believe that @rvagg's (very targeted) fix addresses this issue, I would be shocked if there aren't other race conditions in npm's packing logic. I've already identified a couple other places in the code that are most likely race conditions, even if they're harder to trigger than the current one.
  • The way that dependency bundling is integrated leads to a situation in which a bunch of logic is duplicated between fstream-npm and lib/utils/tar.js in npm itself, and the way fstream's extension mechanism works makes this difficult to clean up. This caused a nasty regression (#13088, see below) as of ~npm@3.8.7 where the dependencies of bundledDependencies were no longer being included in the built package tarballs.
  • The interaction between .npmignore, .gitignore, and files is hopelessly complicated, scattered in many places throughout the code. We've been discussing making the ignores and includes logic clearer and more predictable, and the current code fights our efforts to clean that up.

So, our intention is still to replace fstream, fstream-ignore, and fstream-npm with something much simpler and purpose-built. There's no real reason to have a stream abstraction here when a simple recursive-descent filesystem visitor and a synchronous function that can answer whether a given path should be included in the packed tarball would do the job adequately.

What's not yet clear is whether we'll need to replace node-tar in the process. node-tar is a very robust implementation of tar (it handles, like, everything), and it also includes some very important tweaks to prevent several classes of security exploits involving maliciously crafted packages. However, its packing API involves passing in an fstream instance, so we'd either need to produce something that follows enough of fstream's contract for node-tar to keep working, or swap node-tar out for something like tar-stream (and then ensuring that our use of tar-stream is secure, which could involve security patches for either npm or tar-stream).

The testing and review of fstream@1.0.10 that the team has done leads us to believe that this bug is fixed, but I'm feeling more than a little paranoid about fstream now, so it's important that people keep a close eye on their publishes for a while and let us know immediately if they notice any irregularities.

  • 8802f6c #5082 fstream@1.0.10: Ensure that entries are collected after a paused stream resumes. (@rvagg)
  • c189723 #5082 Remove the warning introduced in npm@3.10.0, because it should no longer be necessary. (@othiym23)

ERK

Because the interaction between fstream, fstream-ignore, fsream-npm, and node-tar is so complex, it's proven difficult to add support for npm features like bundledDependencies without duplicating some logic within npm's code base. While fixing a completely unrelated bug, we "cleaned up" some of this seemingly duplicated code, and in the process removed the code that ensured that the dependencies of bundledDependencies are themselves bundled. We've brought that code back into the code base (without reopening #9642), and added a test to ensure that this regression can't recur.

  • 1b6ceca #13088 Partially restore npm's own version of the fstream-npm function applyIgnores to ensure that the dependencies of bundledDependencies are included in published packages. (@iarna)

GOODBYE, FAITHFUL FRIEND

At NodeConf Adventure 2016 (RIP in peace, Mikeal Rogers's NodeConf!), the CLI team had an opportunity to talk to representatives from some of the larger companies that we knew were still using Node.js 0.8 in production. After asking them whether they were still using 0.8, we got back blank stares and questions like, "0.8? You mean, from four years ago?" After establishing that being able to run npm in their legacy environments was no longer necessary, the CLI team made the decision to drop support for 0.8. (Faithful observers of our team meetings will have known this was the plan for NodeConf since the beginning of 2016.)

In practice, this means only what's in the commit below: we've removed 0.8 from our continuous integration test matrix below, and will no longer be habitually testing changes under Node 0.8. We may also give ourselves permission to use setImmediate() in test code. However, since the project still supports Node.js 0.10 and 0.12, it's unlikely that patches that rely on ES 2015 functionality will land anytime soon.

Looking forward, the team's current plan is to drop support for Node.js 0.10 when its LTS maintenance window expires in October, 2016, and 0.12 when its maintenance / LTS window ends at the end of 2016. We will also drop support for Node.js 5.x when Node.js 6 becomes LTS and Node.js 7 is released, also in the October-December 2016 timeframe.

(Confused about Node.js's LTS policy? Don't be! If you look at this diagram, it should make all of the preceding clear.)

If, in practice, this doesn't work with distribution packagers or other community stakeholders responsible for packaging and distributing Node.js and npm, please reach out to us. Aligning the npm CLI's LTS policy with Node's helps everybody minimize the amount of work they need to do, and since all of our teams are small and very busy, this is somewhere between a necessity and non-negotiable.

  • d6afd5f Remove 0.8 from the Node.js testing matrix, and reorder to match real-world priority, with comments. (@othiym23)

v3.10.0 (2016-06-16):

Do we have a release for you! We have our first new lifecycle since version, a new progress bar and a bunch of bug fixes. I'm really excited about this release, let me tell you!!

DANGER: PUBLISHING ON NODE 6.0.0

Publishing and packing are buggy under Node versions greater than 6.0.0. Please use Node.js LTS (4.4.x) to publish packages. See #5082 for details and current status.

NEW LIFECYCLE SCRIPT: shrinkwrap

  • e8c80f2 #10744 You can now add preshrinkwrap, shrinkwrap and postshrinkwrap to your package.json scripts section. They are run when you run npm shrinkwrap or npm install --save with an npm-shrinkwrap.json present in your module directory.

    preshrinkwrap and shrinkwrap is run prior to generating the new npm-shrinkwrap.json and postshrinkwrap is run after. (@SimenB)

NEW PROGRESS BAR

Install with new progress bar

We have a new progress bar and a bunch of related improvements!

BLOCKING BLOCKING

!!WARNING!! As a part of this change we now explicitly set process.stdout and process.stderr to be blocking if they are ttys, using set-blocking. This is necessary to ensure that we can fully erase the progress bar before we start writing other things out to the console.

Prior to Node.js 6.0.0, they were already blocking on Windows, and MacOS. Meanwhile, on Linux they were always non-blocking but had large (64kb) buffers, which largely made this a non-issue there. Starting with Node.js 6.0.0 they became non-blocking on MacOS and that caused some unexpected issues (see nodejs/node#6456).

If you are a Linux user, it's plausible that this might have a performance impact if your terminal can't keep up with output rate. If you experience this, we want to know! Please file an issue at our issue tracker.

BETTER LAYOUT

Let's start by talking about what goes into the new progress bar:

⸨░░░░░░░░░░⠂⠂⠂⠂⠂⠂⠂⠂⸩ ⠹ loadExtraneous: verb afterAdd /Users/rebecca/.npm/null/0.0.0/package/package.json written
 ↑‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾  ↑ ‾‾‾‾‾‾‾‾‾↑‾‾‾‾   ‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾↑‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾
 percent complete     spinner    current thing we're doing     most recent log line

The spinner is intended as an activity indicator–it moves whenever npm sends something to its logs. It also spins at a constant speed while waiting on the network.

The current thing we're doing relates to how we track how much work has been done. It's the name of the unit of work we most recently started or completed some of. Sometimes these names are more obvious than others and that's something we'll look at improving over time.

And finally, the most recent log line is exactly that, it's the most recent line that you would have seen if you were running with --loglevel=silly or were watching the npm-debug.log. These are written to be useful to the npm developers above all else, so they may sometimes be a little cryptic.

  • 6789978 #13075 npmlog@3.1.2: Update to the latest npmlog, which includes the new and improved progress bar layout. (@iarna)
MORE PERFORMANT

The underlying code for the progress bar was rewritten, in part with performance in mind. Previously whenever you updated the progress bar it would check an internal variable for how long it had been since the last update and if it had been long enough, it would print out what you gave it. With the new progress bar we do updates at a fixed interval (with setInterval) and "updating" the progress bar just updates some variables that will be used when the next tick of the progress bar happens. Currently progress bar updates happen every 50ms, although that's open to tuning.

WIDE(R) COMPATIBILITY

I spent a lot of time working our Unicode support. There were a few issues that plagued us:

Previously one of the characters we used was ambiguous width which means that it was possible to configure your terminal to display it as full width. If you did this, the output would be broken because we assumed it was a half width character. We no longer use any of these characters.

Previously, we defaulted to using Unicode on Windows. This isn't a safe assumption, however, as folks in non-US locales often use other code pages for their terminals. Windows doesn't provide* any facility available to Node.js for determining the current code page, so we no longer try to use Unicode on Windows.

* The facilities it does provide are a command line tool and a windows system call. The former isn't satisfactory for speed reasons and the latter can't be accessed from a JS-only Node.js program.

FOR THE FUTURE: THEMES

The new version of the progress bar library supports plugable themes. Adding support to npm shouldn't be too difficult. The built in themes are:

  • ASCII – The fallback theme which is always available.
  • colorASCII – Inverts the color of the completed portion of the progress bar. The default on Windows and usually on Linux. (Color support is determined by looking at the TERM environment variable.)
  • brailleSpinner – A braille based spinner and other unicode enhancements. MacOS only.
  • colorBrailleSpinner – The default on MacOS, a combination of the above two.
LESS GARBLED OUTPUT

As a part of landing this I've also taken the opportunity to more systematically disable the progress bar prior to printing to stdout or running external commands (in particular: git). This should ensure that the progress bar doesn't get left on screen after something else prints something. We also are now much more zealous about erasing the progress bar on exit, so if you Ctrl-C out of an install we'll still cleanup the progress bar.

  • 63f153c #13075 Consistently make sure that the progress bar is hidden before we try to write to stdout. (@iarna)
  • 8da79fa #13075 Be more methodical about disabling progress bars before running external commands. (@iarna)

REPLACE process.nextTick WITH asap ASAP

  • 5873b56 254ad7e #12754 Use asap in preference over process.nextTick to avoid recursion warnings. Under the hood asap uses setImmediate when available and falls back to process.nextTick when it's not. Versions of node that don't support setImmediate have a version of process.nextTick that actually behaves like the current setImmediate. (@lxe)

FIXES AND REFACTORING

Sometimes the installer would get it into its head that it could move or remove things that it really shouldn't have. While the reproducers for this were often a bit complicated (the core reproducer involved five symlinks(!)), it turns out this is an easy scenario to end up in if your project has a bunch of small modules and you're linking them while developing them.

Fixing this ended up involving doing an important and overdue rewrite of how the installer keeps track of (and interrogates) the relationships between modules. This likely fixes other related bugs, and in the coming weeks we'll verify and close them as we find them. There are a whole slew of commits related to this rewrite, and if you'd like to learn more check out the PR where I describe what I did in detail: #12775

  • 8f3e111 c0b0ed1 #10800 Remove install pruning stage–this was obsoleted by making the installer keep itself up to date as it goes along. This is NOT related to npm prune. (@iarna)

MAKE OUTDATED MORE WIDELY LEGIBLE

  • 21c60e9 #12843 In `npm outdated, stop coloring the Location and Package Type columns. Previously they were colored dark gray, which was hard to read for some users. (@tribou)

DOCUMENTATION UPDATE

  • eb0a72e #12983 Describe how to run the lifecycle scripts of dependencies. How you do this changed with npm v2. (@Tapppi)

DEPENDENCY UPDATES

  • da743dc which@1.2.10: Fix bug where unnecessary special case path handling for Windows could produce unexpected results on Unix systems. (@isaacs)
  • 4533bd5 npm-user-validate@0.1.4: Validate the length of usernames. (@aredridel)
  • 4a18922 glob@7.0.4: Fixes issues with Node 6 and "long or excessively symlink-looping paths". (@isaacs)
  • 257fe11 npm-package-arg@4.2.0: Add escapedName to the result. It is suitable for passing through to a registry without further processing. (@nexdrew)
  • dda3ca7 wrappy@1.0.2 (@zkat)
  • 25f1db5 readable-stream@2.1.4 (@calvinmetcalf)
  • 9d64fe6 abbrev@1.0.9 (@isaacs)

v3.9.6 (2016-06-02):

SMALL OUTPUT TWEAK

  • 0bdc9d1 #12879 The usage output for npm commands was somehow under the impression that the singular form of aliases is aliase. This has been corrected to show alias instead. (@intelliot)

DOC UPDATES

  • f771b49 #12933 Add config.gypi to list of files that are always ignored in the package.json manpage. (@Jokero)

DEPENDENCY UPDATES

v3.9.5 (2016-05-27):

Just a quick point release. We had an issue where I (Kat) included the .nyc_output/ directory in npm@3.9.3 and npm@3.9.4. The issue got reported right after that second release (#12873), and now there's this small point release that's there to fix the issue sooner.

v3.9.4 (2016-05-26):

Hey all! It's that time again!

This week continues our current big-bug squashing push, although there's none that are ready to release quite yet -- we're working on it!

It's also worth noting that we're entering the main part of conference season, so you can probably expect a bit of a dev slowdown as a lot of us wombats attend or speak at the various conferences. Remember npm.camp is happening in 2 months and the lineup is looking pretty great! Tickets are still on sale. Come hang out with us! WOO FUN! 🎉😸

BUGFIX

  • cac0038 #12845 Progress bar during tarball packing now prints pack:packagename instead of pack:[object Object]. (@iarna)

DOC UPDATES

  • 0b81622 #12840 Remove sexualized language from comment in code. (@geek)
  • d6dff24 #12802 Small grammar fix in cli/npm.md. (@andresilveira)
  • cb38e0f #12782 Documents that NOTICE files started getting included after npm/fstream-npm#17. (@SimenB)
  • 70a3ae4 #12776 npm run-script used to have a <pkg> argument that allowed you to target specific packages' scripts. This was removed as one of the breaking changes for npm@2. This patch removes a mention of that argument, which really doesn't exist anymore. (@fibo)

DEP UPDATES

TEST IMPROVEMENTS

So it turns out, t.comment in tap is actually pretty nice! There's also a couple other test improvements by Rebecca landing here.

v3.9.3 (2016-05-19):

This week continues our big-bug squashing adventure! Things are churning along nicely, and we've gotten a lot of fantastic contributions from the community. Please keep it up!

A quick note on last week's release: We had a small npm shrinkwrap-related crasher in npm@3.9.1, so once this release goes out, v3.9.2 is going to be npm@latest. Please update if you ended up in with that previous version!

Remember we have a weekly team meeting, and you can suggest agenda items in the GitHub issue. Keep an eye out for the #npmweekly tag on Twitter, too, and join the conversation! We'll do our best to address questions y'all send us. ✌

FIXES

  • 42d71be #12685 When using npm ls <pkg> without a semver specifier, npm ls would skip any packages in your tree that matched by name, but had a prerelease version in their package.json. This patch fixes it so npm ls does a simple name match unless you use the npm ls <pkg>@<version> format. (@zkat)
  • c698ae6 #12685 Added some tests for more basic npm ls functionality. (@zkat)

NOTABLE DEPENDENCY UPDATES

OTHER DEPENDENCY UPDATES

v3.9.2 (2016-05-17)

This is a quick patch release. The previous release, 3.9.1, introduced a bug where npm would crash given a combination of specific package tree on disk and a shrinkwrap.

  • cde367f #12724 Fix crasher when inflating shrinkwraps with packages on disk that were installed by older npm versions. (@iarna)

v3.9.1 (2016-05-12)

HI all! We have bug fixes to a couple of the hairy corners of npm, in the form of shrinkwraps and bundled dependencies. Plus some documentation improvements and our lodash deps bot a bump.

This is our first week really focused on getting the big bugs list down. Our work from this week will be landing next week, and I can't wait to tell you about that! (It's about symlinks!)

SHRINKWRAP FIX

  • b894413 #12372 Changing a nested dependency in an npm-shrinkwrap.json and then running npm install would not get up the updated package. This corrects that. (@misterbyrne)

BUNDLED DEPENDENCIES FIX

DOCS IMPROVEMENTS

DEPENDENCY UPDATES

v3.9.0 (2016-05-05)

Wow! This is a big release week! We've completed the fixes that let the test suite pass on Windows, plus more general bug fixes we found while fixing things on Windows. Plus a warning to help folks work around a common footgun. PLUS an improvement to how npm works with long cache timeouts.

INFINITE CACHE A LITTLE BETTER

  • 111ae3e #8581 When a package is fetched from the cache which cannot satisfy the version requirements, an attempt to fetch it from the network is made. This is helpful for folks using high values for --cache-min who are willing to accept possibly not-the-most-recent modules in return for less network traffic. (@Zirak)

WARNING: FOOTGUN

  • 60b9a05 #12475 Options can only start with ASCII dashes. Ordinarily this isn't a problem but many web documentation tools "helpfully" convert -- into an emdash (–), or - into an endash (–). If you copy and paste from this documentation your commands won't work the way you expect. This adds a warning that tries to be a little more descriptive about why your command is failing. (@iarna)

WINDOWS CI

We have Windows CI setup now! We still have to tweak it a little bit around paths to the git binaries, but it's otherwise ready!

COVERAGE DATA

Not only do our tests produce coverage reports after they run now, we also automatically update Coveralls with results from Travis CI runs.

EVERYONE BUGS

  • 37c6a51 #12150 Ensure that 'npm cache ls' outputs real filenames. Previously it would sometimes double up the package name in the path it printed. (@isaacs)
  • d3ce0b2 #11444 Fix unbuilding bins for scoped modules. (@iarna)
  • e928a30 #11444 Make handling of local modules (eg npm install /path/to/my/module) more consistent when saved to a package.json. There were bugs previously where it wouldn't consistently resolve relative paths in the same way. (@iarna)
  • b820ed4 #11444 Under certain circumstances the paths produced for linking, either relative or absolute, would end up basing off the wrong virtual cwd. This resulted in failures for npm link in this situations. (@iarna)

WINDOWS BUGS

  • 7380425 #11444 Scoped module names were not being correctly inferred from the path on Windows. (@zkat)
  • 91fc24f #11444 Explore with a command to run didn't work properly in Windows– it would pop open a new cmd window and leave it there. (@iarna)

WINDOWS REFACTORING

  • f07e643 #11444 Move exec path escaping out to its own function. This turns out to be tricky to get right because how you escape commands to run on Windows via cmd is different then how you escape them at other times. Specifically, you HAVE to quote each directory segment that has a quote in it, that is: C:\"Program Files"\MyApp\MyApp.exe By contrast, if that were an argument to a command being run, you CAN'T DO quote it that way, instead you have to wrap the entire path in quotes, like so: "C:\Program Files\MyApp\MyApp.exe". (@iarna)
  • 2e01d29 #11444 Create a single function for detecting if we're running on Windows (and using a Windows shell like cmd) and use this instead of doing it one-off all over the place. (@iarna)

FIX WINDOWS TESTS

As I said before, our tests are passing on Windows! 🎉

DEPENDENCY UPDATES

v3.8.9 (2016-04-28)

Our biggest news this week is that we got the Windows test suite passing! It'll take a little longer to get it passing in our Windows CI but that's coming soon too.

That means we'll be shifting gears away from tests to fixing Big Bugs™ again. Join us at our team meeting next Tuesday to learn more about that.

BUG FIXES AND REFACTORING

  • 60da618 #12347 Fix a bug that could result in shrinkwraps missing the resolved field, which is necessary in producing a fully reproducible build. (@sminnee)

  • 8597ba4 #12009 Fix a bug in npm view <packagename> versions that resulted in bad output if you didn't also pass in --json. (@watilde)

  • 20125f1 a53feac 6cfbae4 #12485 Refactor how the help summaries for commands are produced, such that we only have one list of command aliases. (@watilde)

  • 2ae210c read-package-json@2.0.4: Fix a crash we discovered while fixing up the Windows test suite where if you had a file in your node_modules it would cause a crash on Windows (but not MacOS/Linux).

    This makes the error code you get on Windows match that from MacOS/Linux if you try to read a package.json from a path that includes a file, not a folder. (@zkat)

v3.8.8 (2016-04-21)

Hi all! Long time no see! We've been heads-down working through getting our test suite passing on Windows. Did you know that we have Windows CI now running over at Appveyor? In the meantime, we've got a bunch of dependency updates, some nice documentation improvements and error messages when your package.json contains invalid JSON. (Yeah, I thought we did that last one before too!)

BAD JSON IS BAD

DOCUMENTATION

  • 7d64301 #12415 Clarify that when configuring client-side certificates for authenticating to non-npm registries that cert and key are not filesystem paths and should actually include the certificate and key data. (@rvedotrc)
  • f8539b8 #12324 Describe how npm run sets NODE and PATH in more detail. Note that npm run changes PATH to include the current node interpreter’s directory. (@addaleax)
  • 2b57606 #11461 Clarify the documentation for the package.json homepage field. (@stevemao)

TESTS

  • b5a0fbb #12329 Fix progress config testing to ignore local user configs. Previously, any local setting would cause the tests to fail as they were trying to test what the default values for the progress bar would be in different environments and any explicit setting overrides those defaults. (@iarna)
  • 3d195bc The lifecycle-signal test could crash on v0.8 due to its use of Number.parseInt, which isn't available in that version of node. Fortunately global.parseInt is, so we just use that instead. (@iarna)

DEPENDENCY UPDATES

  • 05a28e3 npm-package-arg@4.1.1: Under some circumstances file:// URLs on Windows were not handled correctly.

    Also, stop converting local module/tarballs into full paths in this module. We do already do that in realize-package-specifier, which is more appropriate as it knows what package we're installing relative to. (@zkat)

  • ada2e93 realize-package-specifier@3.0.3: Require the new npm-package-arg, plus fix a case where specifiers that were maybe a tag, maybe a local filename were resolved differently than those that were definitely a local filename. (@zkat) (@iarna)

  • adc515b fs-vacuum@1.2.9: A fix for AIX where a non-empty directory can cause fs.rmDir to fail with EEXIST instead of ENOTEMPTY and three new tests (@richardlau)

    Code cleanup, CI & dependency updates. (@othiym23)

  • ef53a46 tap@5.7.1 (@isaacs)

  • df1f2e4 request@2.72.0: Fix crashes when response headers indicate gzipped content but the body is empty. Add support for the deflate content encoding. (@simov)

  • 776c599 readable-stream@2.1.0: Adds READABLE_STREAM env var that, if set to disable, will make readable-stream use the local native node streams instead. (@calvinmetcalf)

  • 10d6d55 normalize-git-url@3.0.2: Add support git+file:// type URLs. (@zkat)

  • 75017ae lodash.union@4.3.0 (@jdalton)

v3.8.7 (2016-04-07)

IMPROVED DIAGNOSTICS

  • 38cf79f #12083 If you ignore-scripts to disable lifecycles, this makes npm report when it skips running a script. (@bfred-it)

IMPROVE AUTO-INCLUDES

  • c615182 #11995 There were bugs where modules whose names matched the special files that npm always includes would be included, for example, the history package was always included.

    With npm@3 such extraneously bundled modules would not be ordinarily used, as things in node_modules in packages are ignored entirely if the package isn't marked as bundling modules.

    Because of this npm@3 behavior, the files-and-ignores test failed to catch this as it was testing install output not what got packed. That has also been fixed. (@glenjamin)

DOCUMENTATION UPDATES

DEPENDENCY UPDATES

v3.8.6 (2016-03-31)

Heeeeeey y'all.

Kat here! Rebecca's been schmoozing with folks at Microsoft Build, so I'm doing the npm@3 release this week.

Speaking of Build, it looks like Microsoft is doing some bash thing. This might be really good news for our Windows users once it rolls around. We're keeping an eye out and feeling hopeful. 🙆

As far as the release goes: We're really happy to be getting more and more community contributions! Keep it up! We really appreciate folks trying to help us, and we'll do our best to help point you in the right direction. Even things like documentation are a huge help. And remember -- you get socks for it, too!

FIXES

DOC UPDATES

DEP BUMPS

v3.8.5 (2016-03-24)

Like my esteemed colleague @zkat said in this week's LTS release notes, this week is another small release but we are continuing to work on our Windows efforts.

You may also be interested in reading the LTS process and policy that @othiym23 put together recently. If you have any feedback, we would love to hear.

DOCTOR IT HURTS WHEN LINK TO MY LINK

Well then, don't do that.

ERR MODULE LIST TOO LONG

  • b271ed2 #11983 Exit early if no arguments were provided to search instead of trying to display all the modules, running out of memory, and then crashing. (@SimenB)

ELIMINATE UNUSED MODULE

DOCUMENTATION IMPROVEMENTS

  • fdd6b28 #11884 Include node_modules in the list of files and directories that npm won't include in packages ordinarily. (Modules listed in bundledDependencies and things that those modules rely on, ARE included of course.) (@Jameskmonger)
  • aac15eb #12006 Fix typo in npm-orgs documentation, where teams docs went to access docs and vice versa. (@yaelz)

FEWER NETWORK TESTS

v3.8.4 (2016-03-24)

Was erroneously released with just a changelog typo correction and was otherwise the same as 3.8.3.

v3.8.3 (2016-03-17):

SECURITY ADVISORY: BEARER TOKEN DISCLOSURE

This release includes the fix for a vulnerability that could cause the unintentional leakage of bearer tokens.

Here are details on this vulnerability and how it affects you.

DETAILS

Since 2014, npm’s registry has used HTTP bearer tokens to authenticate requests from the npm’s command-line interface. A design flaw meant that the CLI was sending these bearer tokens with every request made by logged-in users, regardless of the destination of their request. (The bearers only should have been included for requests made against a registry or registries used for the current install.)

An attacker could exploit this flaw by setting up an HTTP server that could collect authentication information, then use this authentication information to impersonate the users whose tokens they collected. This impersonation would allow them to do anything the compromised users could do, including publishing new versions of packages.

With the fixes we’ve released, the CLI will only send bearer tokens with requests made against a registry.

THINK YOU'RE AT RISK? REGENERATE YOUR TOKENS

If you believe that your bearer token may have been leaked, invalidate your current npm bearer tokens and rerun npm login to generate new tokens. Keep in mind that this may cause continuous integration builds in services like Travis to break, in which case you’ll need to update the tokens in your CI server’s configuration.

WILL THIS BREAK MY CURRENT SETUP?

Maybe.

npm’s CLI team believes that the fix won’t break any existing registry setups. Due to the large number of registry software suites out in the wild, though, it’s possible our change will be breaking in some cases.

If so, please file an issue describing the software you’re using and how it broke. Our team will work with you to mitigate the breakage.

CREDIT & THANKS

Thanks to Mitar, Will White & the team at Mapbox, Max Motovilov, and James Taylor for reporting this vulnerability to npm.

PERFORMANCE IMPROVEMENTS

The updated are-we-there-yet changes how it tracks how complete things are to be much more efficient. The summary is that are-we-there-yet was refactored to remove an expensive tree walk.

The result for you should be faster installs when working with very large trees.

Previously are-we-there-yet computed this when you asked by passing the request down its tree of progress indicators, totaling up the results. In doing so, it had to walk the entire tree of progress indicators.

By contrast, are-we-there-yet now updates a running total when a change is made, bubbling that up the tree from whatever branch made progress. This bubbling was already going on so there was nearly no cost associated with taking advantage of it.

  • 32f2bd0 npmlog@2.0.3: Bring in substantial performance improvements from are-we-there-yet. (@iarna)

DUCT TAPE FOR BUGS

  • 473d324 #11947 Guard against bugs that could cause the installer to crash with errors like:

    TypeError: Cannot read property 'target' of null

    This doesn't fix the bugs, but it does at least make the installer less likely to explode. (@thefourtheye)

DOC FIXES

DEPENDENCY UPDATES

  • 7537fe1 sorted-object@2.0.0: Create objects with {} instead of Object.create(null) to make the results strictly equal to what, say, parsed JSON would provide. (@domenic)
  • 8defb0f readable-stream@2.0.6: Fix sync write issue on 0.10. (@calvinmetcalf)

TEST FIXES FOR THE SELF TESTS

  • c3edeab #11912 Change the self installation test to do its work in /tmp. Previously this was installing into a temp subdir in test/tap, which wouldn't catch the case where a module was installed in the local node_modules folder but not in dependencies, as node would look up the tree and use the copy from the version of npm being tested. (@iarna)

v3.8.2 (2016-03-10):

HAVING TROUBLE INSTALLING C MODULES ON ANDROID?

This release includes an updated node-gyp with fixes for Android.

NPM LOGOUT CLEANS UP BETTER

  • 460ed21 #10529 If you ran npm logout with a scope, while we did invalidate your auth token, we weren't removing the auth token from your config file. This patch causes the auth token to be removed. (@wyze)

HELP MORE HELPFUL

  • d1d0233 #11003 Update help to only show command names and their shortcuts. Previously some typo corrections were shown, along with various alternate spellings. (@watilde)
  • 47928cd #11003 Remove "version" typo from the help listing. (@doug-wade)

MORE COMPLETE CONFIG LISTINGS

DEPTH LIMITED PARSEABLE DEP LISTINGS

PROGRESS FOR THE (NON) UNICODE REVOLUTION

npm view --json, NOW ACTUALLY JSON

  • 24ab70a #11808 Make npm view produce valid JSON when requested with --json. Previously npm view produced some sort of weird hybrid output, with multiple JSON docs. (@doug-wade)

DOCUMENTATION CHANGES

  • 6fb0499 #11726 Previously we patched the npm update docs to suggest using --depth Infinity instead of --depth 9999, but that was a mistake. We forgot that npm outdated (on which npm update is built) has a special case where it treats Infinity as 0. This reverts that patch. (@GriffinSchneider)
  • f0bf684 #11748 Document all of the various aliases for commands in the documentation for those commands. (@watilde)
  • fe04443 #10968 The npm-scope document notes that scopes have been available on the public registry for a while. This adds that you'll need npm@2 or later to use them. (@doug-wade)
  • 3db37a5 #11820 The command npm link should be linking package from local folder to global, and npm link package-name should be from global to local. The description in the documentation was reversed and this fixes that. (@rhgb)

GLOB FOR THE GLOB THRONE

  • be55882 glob@7.0.3: Fix a race condition and some windows edge cases. (@isaacs)

v3.8.1 (2016-03-03):

This week the install summary got better, killing your npm process now also kills the scripts it was running and a rarely used search flag got documented.

Our improvements on the test suite on Windows are beginning to pick up steam, you can follow along by watching the PR.

BETTER INSTALL SUMMARIES

  • e40d457 #11699 Ensure that flags like --production passed to install don't result in the summary at the end being incorrectly filtered. That summary is produced by the same code as npm ls and therefore responds to flags the same way it does. This is undesirable when it's an install summary, however, as we don't want it to filter anything.

    This fixes an issue where npm install --production <module> would result in npm exiting with an error code. The --production flag would make npm ls filter out <module> as it wasn't saved to the package.json and thus wasn't a production dependency. The install report is limited to show just the modules installed, so with that filtered out nothing is available. With nothing available npm ls would set npm to exit with an error code. (@ixalon)

  • 99337b4 #11600 Make the report of installed modules really only show those modules that were installed. Previously it selected which modules from your tree to display based on name@version which worked great when your tree was deduped but would list things it hadn't touched when there were duplicates. (@iarna)

SCRIPTS BETTER FOLLOW THE LEADER

  • 5454347 #10868 When running a lifecycle script, say through npm start, killing npm wouldn't forward that on to the children. It does now. (@daniel-pedersen)

SEARCHING SPECIFIC REGISTRIES

LODASH UPDATES

v3.8.0 (2016-02-25):

This week brings a quality of life improvement for some Windows users, and an important knob to be tuned for folks experiencing network problems.

LIMIT CONCURRENT REQUESTS

We've long known that npm's tendency to try to request all your dependencies simultaneously upset some network hardware (particular, consumer grade routers & proxies of all sorts). One of the reasons that we're planning to write our own npm specific version of request is to be able to more easily control this sort of thing.

But fortunately, you don't have to wait for that. @misterbyrne took a look at our existing code and realized it could be added painlessly TODAY. The new default maximum is 50, instead of Infinity. If you're having network issues you can try setting that value down to something lower (if you do, please let us know... the default is subject to tuning).

WINDOWS GIT BASH

We think it's pretty keen too, we were making it really hard to actually upgrade if you were using it. NO MORE!

  • d60351c #11524 Prefer locally installed npm in Git Bash -- previous behavior was to use the global one. This was done previously for other shells, but not for Git Bash. (@destroyerofbuilds)

DOCUMENTATION IMPROVEMENTS

DEPENDENCY UPDATES

v3.7.5 (2016-02-22):

A quick fixup release because when I updated glob, I missed the subdep copies of itself that it installed deeper in the tree. =/

This only effected people trying to update to 3.7.4 from npm@2 or npm@1. Updates from npm@3 worked fine (as it fixes up the missing subdeps during installation).

OH MY GLOB

  • 63fa704 #11633 When updating the top level npm to glob@7, the subdeps that still depended on glob@6 got new versions installed but they weren't added to the commit. This adds them back in. (@iarna)

v3.7.4 (2016-02-18):

I'm (@iarna) back from vacation in the frozen wastes of Maine! This release sees a couple of bug fixes, some documentation updates, a bunch of dependency updates and improvements to our test suite.

FIXES FOR update, FIXES FOR ls

@wyze, DOCUMENTATION HERO OF THE PEOPLE, GETS THEIR OWN HEADER

WHITTLING AWAY AT PATH LENGTHS

So for all of you who don't know -- Node.js does, in fact, support long Windows paths. Unfortunately, depending on the tool and the Windows version, a lot of external tooling does not. This means, for example, that some (all?) versions of Windows Explorer can literally never delete npm from their system entirely because of deeply-nested npm dependencies. Which is pretty gnarly.

Incidentally, if you run into that in particularly, you can use rimraf to remove such files 💁.

The latest victim of this issue was the Node.js CI setup for testing on Windows, which uses some tooling or another that croaks on the usual path length limit for that OS: 255 characters.

This isn't ordinarily an issue with npm@3 as it produces mostly flat trees, but you may be surprised to learn that npm's own distribution isn't flat, due to needing to be compatible with npm@1.2, which ships with node@0.8!

We've taken another baby step towards alleviating this in this release by updating a couple of dependencies that were preventing npmlog from deduping, and then doing a dedupe on that and gauge. Hopefully it helps.

  • f3c32bc #11528 node-gyp@3.3.0: Update to a more recent version that uses a version of npmlog compatible with npm itself. Also adds: AIX support, new gyp, --cafile command line option, and allows configuration of Node.js and io.js mirrors. (@rvagg)

INTERNAL TEST IMPROVEMENTS

The npm core team's time recently has been sunk into npm's many years of tech debt. Specifically, we've been working on improving the test suite. This isn't user visible, but in future should mean a more stable, easier to contribute to npm. Ordinarily we don't report these kinds of changes in the change log, but I thought I might share this week as this chunk is bigger than usual.

  • 07f020a #11292 tacks@1.0.9: Add a package that provides a tool to generate fixtures from folders and, relatedly, a module that an create and tear down filesystem fixtures easily. (@iarna)
  • 0837346 #11292 Remove all the relatively cryptic legacy tests and creates new tap tests that check the same functionality. The legacy tests were tests that were originally a shell script that was ported to javascript early in npm's history. (@iarna) (@zkat)
  • 5a701e7 #11292 Test that we don't leak auth info into the environment. (@zkat)
  • 502d7d0 #11292 Test that env vars properly passed into scripts. (@zkat)
  • 420f267 #11292 Test that npm's distribution binary is complete and can be installed and used. (@iarna)
  • b7e99be #11292 Test that the package.json files section and .npmignore do what they're supposed to. (@zkat)

DEPENDENCY UPDATES

v3.7.3 (2016-02-11):

Hey all! We've got a pretty small release this week -- just documentation updates and a couple of dependencies. This release also includes a particular dependency upgrade that makes it so we're exclusively using the latest version of graceful-fs, which'll make it so things keep working with future Node.js releases.

A certain internal Node.js API was deprecated and slated for future removal from Node Core. This API was critical for versions of graceful-fs@<4, before a different approach was used to achieve similar ends. By upgrading this library, and making sure all our dependencies are also updated, we've ensured npm will continue to work once the API is finally removed. Older versions of npm, on the other hand, will simply not work on future versions of Node.js.

DEPENDENCY UPGRADES

EVERYONE GETTING SOCKS LIKE IT'S OPRAH'S SHOW

  • 9ea5658 #11410 Fixed a small spelling error in npm-config.md. (@pra85)
  • 2a11e56 #11403 Removes --depth Infinity warning from documentation -- this operation should actually be totally safe as of npm@3. (The warning remains for npm@2.) (@Aourin)
  • 42a4727 #11391 Fixed versions of shrinkwrap.json in examples in documentation for npm shrinkwrap, which did not quite match up. (@xcatliu)

v3.7.2 (2016-02-04):

This week, the CLI team has been busy working on rewriting tests to support getting coverage reports going and running all of our tests on Windows. Meanwhile, we've got a bunch of dependency updates and one or two other things.

TESTS WENT INTO HIDING

Last week we took a patch from @substack to stop the installer from reordering arrays in an installed module's package.json... but somehow I dropped the test when I was rebasing.

  • 21b9271 #10063 Restore test that verifies that we don't re-order arrays in a module's package.json on install. (@substack)

DOCUMENTATION FIXES

  • c67521d #11348 Improve the documentation around which files are ALWAYS included in published packages and which are ALWAYS excluded. (@jscissr)
  • 7ef6793 #11348 The release date on the 3.7.0 changelog entry was wrong. I honestly don't know how I keep doing this. =D (@rafek)

DEPENDENCY UPDATES

  • 8a3c80c graceful-fs@4.1.3: Fix a bug where close wasn't getting made graceful. (@isaacs)

lodash saw updates across most of its modules this week with browser campatibility fixes that don't really impact us.

v3.7.1 (2016-02-01):

Super quick Monday patch on last week's release.

If you ever wondered why we release things to the npm@next tag for a week before promoting them to npm@latest, this is it!

RELEASE TRAIN VINDICATED (again)

  • adcaf04 #11349 Revert last weeks change to use JSON clone instead of lodash.cloneDeep. (@iarna)

v3.7.0 (2016-01-29):

Hi all! This week brings us some important performance improvements, support for git submodules(!) and a bunch of bug fixes.

PERFORMANCE

gauge, the module responsible for drawing npm's progress bars, had an embarrassing bug in its debounce implementation that resulted in it, on many systems, actually being slower than if it hadn't been debouncing. This was due to it destroying and then creating a timer object any time it got an update while waiting on its minimum update period to elapse. This only was a measurable slowdown when sending thousands of updates a second, but unfortunately parts of npm's logging do exactly that. This has been patched to eliminate that churn, and our testing shows the progress bar as being eliminated as a source of slow down.

Meanwhile, are-we-there-yet is the module that tracks just how complete our big asynchronous install process is. @STRML spent some time auditing its source and made a few smaller performance improvements to it. Most impactful was eliminating a bizarre bit of code that was both binding to AND closing over the current object. I don't have any explanation for how that crept in. =D

  • c680fa9 npmlog@2.0.2: New are-we-there-yet with performance patches from @STRML. New gauge with timer churn performance patch. (@iarna)

We were also using lodash's cloneDeep on package.json data which is definitely overkill, seeing as package.json data has all the restrictions of being json. The fix for this is just swapping that out for something that does a pair of JSON.stringify/JSON.parse, which is distinctly more speedy.

NEW FEATURE: GIT SUBMODULE SUPPORT

Long, long requested– the referenced issue is from 2011– we're finally getting rudimentary git submodule support.

  • 39dea9c #1876 Add support for git submodules in git remotes. This is a fairly simple approach, which does not leverage the git caching mechanism to cache submodules. It also doesn't provide a means to disable automatic initialization, e.g. via a setting in the .gitmodules file. (@gagern)

ROBUSTNESS

  • 5dec02a #10347 There is an obscure feature that lets you monkey-patch npm when it starts up. If the module being required with this feature failed, it would previously just make npm error out– this reduces that to a warning. (@evanlucas)

BUG FIXES

  • 9ab8b8d #10820 Fix a bug with npm ls where if you asked for ONLY production dependencies in output it would exclude dependencies that were BOTH production AND development dependencies. (@davidvgalbraith)
  • 6803fed #8982 Fix a bug where, under some circumstances, if you had a path that contained the name of a package being installed somewhere in it, npm would incorrectly refuse to run lifecycle scripts. (@elvanja)
  • 3eae40b #9253 Fix a bug where, when running lifecycle scripts, if the Node.js binary you ran npm with wasn't in your PATH, npm wouldn't use it to run your scripts. (@segrey)
  • 61daa6a #11014 Fix a bug where running rimraf node_modules/<package> followed by npm rm --save <package> would fail. npm now correctly removes the module from your package.json even though it doesn't exist on disk. (@davidvgalbraith)
  • a605586 #9679 Fix a bug where npm install --save git+https://… would save a https:// url to your package.json which was a problem because npm wouldn't then know that it was a git repo. (@gagern)
  • bbdc700 #10063 Fix a bug where npm would change the order of array properties in the package.json files of dependencies. npm adds a bunch of stuff to package.json files in your node_modules folder for debugging and bookkeeping purposes. As a part of this process it sorts the object to reduce file churn when it does updates. This fixes a bug where the arrays in the object were also getting sorted. This wasn't a problem for properties that npm itself maintains, but is a problem for properties used by other packages. (@substack)

DOCS IMPROVEMENTS

  • 2609a29 #11273 Include an example of viewing package version history in the npm view documentation. (@vedatmahir)
  • 719ea9c #11272 Fix typographical issue in npm update documentation. (@jonathanp)
  • cb9df5a #11215 Do not call SEE LICENSE IN <filename> an SPDX expression, as it's not. (@kemitchell)
  • f427934 #11196 Correct the package.json examples in the npm update documentation to actually be valid JSON and not just JavaScript object literals. (@s100)

DEPENDENCY UPDATES

  • a7b2407 retry@0.9.0: New features and interface agnostic refactoring. (@tim-kos)
  • 220fc77 request@2.69.0: A bunch of small bug fixes and module updates. (@simov)
  • 9e5c84f which@1.2.4: Update isexe and fix bug in pathExt, in which files without extensions would sometimes be preferred to files with extensions on Windows, even though those without extensions aren't executable. pathExt is a list of extensions that are considered executable (exe, cmd, bat, com on Windows). (@isaacs)
  • 375b9c4 rimraf@2.5.1: Minor doc formatting fixes. (@isaacs)
  • ef1971e lodash.clonedeep@4.0.2: Misc minor code cleanup. No functional changes. (@jdalton)

v3.6.0 (2016-01-20):

Hi all! This is a bigger release, in part 'cause we didn't have one last week. The most important thing you need to know is that when npm@3.6.0 replaces npm@3.5.4 as next, npm@3.5.4 WILL NOT be moved on to latest. This is due to a packaging error that tickles bugs in some earlier releases and makes upgrades to it from those versions break the install.

NEW FEATURES‼

  • ff504d4 #8752 In npm outdated, report symlinked packages as having a wanted & latest version of linked. (@halhenke)

  • f44d8c9 #10775 Add a success message to adduser / login. (@ekmartin)

  • 3109303 #10043 Warn if you try to use npm run x if you don't have a node_modules folder, since whatever you're trying to do probably won't work. (@timkrins)

  • 9ed2849 e9f1ad8 f10d300 8b593d8 #10717 npm version can now take a from-git argument, which instructs npm to read the version from git and update your package.json to what it finds. This is in contrast to its normal use where npm tells git about your new version. (@ekmartin)

3.5.4 WAS NOT SO GREAT

The npm@3.5.4 package was missing some dependencies. Specifically, glob and has-unicode had major release updates which meant that subdeps that relied on older major versions couldn't use the npm supplied versions any more, and so they needed their own copies.

This went undetected because the actions necessary to run the tests (which check for this sort of thing) resolved the missing modules.

Further, it didn't have symptoms when upgrading from most versions of npm. Unfortunately, some versions had bugs that were tickled by this and resulted in broken upgrades, most notably, npm@3.3.12, the version that's been in Node.js 5.

WHEN MISSING PATHS ARE OK

  • bb638fa #11212 When trying to determine if a file was controlled by npm before going to remove it, we check to see if it is inside any of a list of paths that npm considers to be under its control. Not all of those paths always exist (and that's ok!) Previously we were calling it a failure to match if ANY of them didn't exist. We now only do so if NONE of them exist. If some do, then we do our usual checks on them.

    This showed up as an error where you would see something like:

    npm warn gentlyRm not removing /path/to/thing as it wasn't installed by /path/to/other/thing

    But it totally was installed by it. (@iarna)

BETTER NODE PRE-RELEASE SUPPORT

Historically, if you used a pre-release version of Node.js, you would get dozens and dozens of warnings when EVERY engine check failed across all of your modules, because >= 0.10.0 doesn't match prereleases.

You might find this stream of redundant warnings undesirable. I do.

We've moved this into a SINGLE warning you'll get about using a pre-release version of Node.js and now suppress those other warnings.

  • 6952f79 #11212 Engine check warnings are now issued along with any other warnings about your tree, instead of emitting in the middle of your install (and then disappearing behind the giant tree of stuff installed). (@iarna)
  • ee2ebe9 #11212 Suppress engine verification warnings about pre-release versions of Node.js. (@iarna)
  • 135b7e0 #11212 Explicitly warn, in only one place, if you are using a pre-release version of Node.js. (@iarna)

BUG FIXES

  • ea331c8 #10938 When removing a package, sometimes the node_modules/.bin wouldn't be cleaned up entirely. This would result in package folders that contained only a node_modules/.bin directory. In turn, this would result in npm ls and other tools complaining about these broken directories. To fix this, the unbuild step now explicitly deletes the node_modules/.bin folder as its final step. (@chrisirhc)
  • 00720db #11158 On Windows, the node-gyp wrapper would fail if your path to node-gyp contained spaces. This fixes that problem by quoting use of that path. (@orangemocha)
  • 69ac933 #11142 Fix a race condition when making directories in the cache, which could lead to ENOENT failures. (@Jimbly)
  • e982858 #9696 When replacing the package.json in the cache you sometimes see EPERM errors on Windows that you wouldn't on Unix-like operating systems. This ignores those errors and allows Windows to continue. Longer term, we'll be adding something to retry these errors, but ultimately fail if there really is an ongoing permissions issue. (@orangemocha)

DOC CHANGES

DEPENDENCY UPDATES

v3.5.4 (2016-01-07):

I hope you all had fantastic winter holidays, if it's winter where you are and if there are holidays‼ We went a few weeks without releases because staff was taking time away from work here and there. A new year has come and we're back now, and refreshed and ready to dig in!

This week brings us a bunch of documentation improvements and some module updates. The core team's focus continues to be on improving tests, particularly with Windows, so there's not too much to call out here.

DOCUMENTATION IMPROVEMENTS

  • 6b0031e #11044 Correct documentation regarding the defaults for the color config option. (@scottaddie)
  • c6ce69e #10990 Drop mentions in documentation of process.installPrefix, as it hasn't been a thing since Node.js 0.6 and we don't support that. (@jeffmcmahan)
  • dee92d1 #11037 Clarify the documentation on the max length of the name property in package.json files. (@scottaddie)
  • 4b9d7bb #10787 Make the formatting in the documentation for npm dist-tag more consistent with other docs. (@cvrebert)
  • 7f77a80 #10787 Add documentation to the npm dist-tag docs that explains in greater detail how latest is different than other tags. Further, improve the documentation with better examples. Add a discussion of common practice for using dist tags to manage alpha's and beta's. (@cvrebert)
  • 6db58dd 2ee6371 #10788 #10789 Improve documentation cross referencing. (@cvrebert)
  • 7ba629a #10790 Document more clearly that npm install foo means npm install foo@latest. (@cvrebert)

A FEW MODULE UPDATES

  • fc2e8d5 glob@6.0.3: Remove deprecated features and fix a bunch of bugs. (@isaacs)
  • 5b820c4 has-unicode@2.0.0: Change the default on Windows to be false, as international Windows installs often install to non-unicode codepages and there's no way to detect this short of a system call or a call to a command line program. (@iarna)
  • 238fe84 which@1.2.1: Fixed bugs with uid/gid checks and with quoted Windows PATH parts. (@isaacs)
  • 5e510e1 rimraf@2.5.0: Add ability to disable glob support / pass in options. (@isaacs)
  • 7558215 readable-stream@2.0.5: Minor performance improvements. (@calvinmetcalf)
  • 64e8499 fs-write-stream-atomic@1.0.8: Rewrite to use modern streams even on 0.8 plus a bunch of tests. (@iarna)
  • 74d92a0 columnify@1.5.4: Some bug fixes around large inputs. (@timoxley)

FIX NPM'S TESTS ON 0.8

This doesn't impact you as a user of npm, and ordinarily that means we wouldn't call it out here, but if you've ever wanted to contribute, having that green travis badge makes it a lot easier to do so with confidence!

  • b14cdbb #10872 Rewrite tests using nock to use other alternatives. (@zkat)

  • 59ed01a #10872 Work around Node.js 0.8 http back-pressure bug.

    0.8 http streams have a bug, where if they're paused with data in their buffers when the socket closes, they call end before emptying those buffers, which results in the entire pipeline ending and thus the point that applied backpressure never being able to trigger a resume.

    We work around this by piping into a pass through stream that has unlimited buffering. The pass through stream is from readable-stream and is thus a current streams3 implementation that is free of these bugs even on 0.8. (@iarna)

v3.5.3 (2015-12-10):

Did you know that Bob Ross reached the rank of master sergeant in the US Air Force before becoming perhaps the most soothing painter of all time?

TWO HAPPY LITTLE BUG FIXES

  • 71c9590 #10505 npm ls --json --depth=0 now respects the depth parameter, when it is zero and when it is not zero. (@MarkReeder)
  • 954fa67 #9099 I had always thought you could run npm version from subdirectories in your project, which is great, because now you can. I guess I was just ahead of my time. (@ekmartin)

NOW PAINT IN SOME NICE DOCS CHANGES

  • b88c37c #10546 Goodbye, FAQ! You were cheeky and fun until you weren't! Don't worry: npm still loves everyone, especially you! (@ashleygwilliams)
  • 2d3afe9 #10570 Update documentation URLs to be HTTPS everywhere sensible. No HTTP shall be spared! (@rsp)
  • 6abd0e0 #10650 Correctly note that there are two lifecycle scripts run by an install phase in an example, instead of three. (@eymengunay)
  • a5e8df5 #10687 npm outdated's output can be a little puzzling sometimes. I've attempted to make it clearer, with some examples, of what's going on with "wanted" and "latest" in more cases. (@othiym23)
  • 8f52833 #10700 Hey, do you remember when search.npmjs.org was a thing? I think I do? The last time I used it was in like 2012, and it's gone now, so remove it from the docs. (@gagern)
  • b6a53b8 npm/docs#477 Continue to airbrush the CLI API docs out of history. (@verpixelt)
  • b835b72 semver@5.1.0: Include BNF for SemVer expression grammar (which is also now included in npm help semver). (@isaacs)

LAND YOUR DEPENDENCY UPGRADES IN PAIRS SO EVERYONE HAS A FRIEND

v3.5.2 (2015-12-03):

Weeeelcome to another npm release! The short version is that we fixed some ENOENT and some modules that resulted in modules going missing. We also eliminated the use of MD5 in our code base to help folks using Node.js in FIPS mode. And we fixed a bad URL in our license file.

FIX URL IN LICENSE

The license incorrectly identified the registry URL as registry.npmjs.com and this has been corrected to registry.npmjs.org.

ENOENT? MORE LIKE ENOMOREBUGS

The headliner this week was uncovered by the fixes to bundled dependency handling over the past few releases. What had been a frustratingly intermittent and hard to reproduce bug became something that happened every time in Travis. This fixes another whole bunch of errors where you would, while running an install have it crash with an ENOENT on rename, or the install would finish but some modules would be mysteriously missing and you'd have to install a second time.

What's going on was a bit involved, so bear with me:

npm@3 generates a list of actions to take against the tree on disk. With the exception of lifecycle scripts, it expects these all to be able to act independently without interfering with each other.

This means, for instance, that one should be able to upgrade b in a→b→c without having npm reinstall c.

That works fine by the way.

But it also means that the move action should be able to move b in a→b→c@1.0.1 to a→d→b→c@1.0.2 without moving or removing c@1.0.1 and while leaving c@1.0.2 in place if it was already installed.

That is, the move action moves an individual node, replacing itself with an empty spot if it had children. This is not, as it might first appear, something where you move an entire branch to another location on the tree.

When moving b we already took care to leave c@1.0.1 in place so that other moves (or removes) could handle it, but we were stomping on the destination and so c@1.0.2 was being removed.

There was also a bug with remove where it was pruning the entire tree at the remove point, prior to running moves and adds.

This was fine most of the time, but if we were moving one of the deps out from inside it, kaboom.

  • 19c626d #10655 Get rid of the remove commit phase– we could have it prune just the module being removed, but that isn't gaining us anything. (@iarna)

After all that, we shouldn't be upgrading the add of a bundled package to a move. Moves save us from having to extract the package, but with a bundled dependency it's included in another package already so that doesn't gain us anything.

While I was in there, I also took some time to improve diagnostics to make this sort of thing easier to track down in the future:

NO MORE MD5

We updated modules that had been using MD5 for non-security purposes. While this is perfectly safe, if you compile Node in FIPS-compliance mode it will explode if you try to use MD5. We've replaced MD5 with Murmur, which conveys our intent better and is faster to boot.

DEPENDENCY UPDATES

v3.5.1 (2015-11-25):

THE npm CLI !== THE npm REGISTRY !== npm, INC.

npm-the-CLI is licensed under the terms of the Artistic License 2.0, which is a liberal open-source license that allows you to take this code and do pretty much whatever you like with it (that is, of course, not legal language, and if you're doing anything with npm that leaves you in doubt about your legal rights, please seek the review of qualified counsel, which is to say, not members of the CLI team, none of whom have passed the bar, to my knowledge). At the same time the primary registry the CLI uses when looking up and downloading packages is a commercial service run by npm, Inc., and it has its own Terms of Use.

Aside from clarifying the terms of use (and trying to make sure they're more widely known), the only recent changes to npm's licenses have been making the split between the CLI and registry clearer. You are still free to do whatever you like with the CLI's source, and you are free to view, download, and publish packages to and from registry.npmjs.org, but now the existing terms under which you can do so are more clearly documented. Aside from the two commits below, see also the release notes for npm@3.4.1, which is where the split between the CLI's code and the terms of use for the registry was first made more clear.

  • 35a5dd5 #10532 Clarify that registry.npmjs.org is the default, but that you're free to use the npm CLI with whatever registry you wish. (@kemitchell)
  • fa6b013 #10532 Having semi-duplicate release information in README.md was confusing and potentially inaccurate, so remove it. (@kemitchell)

EASE UP ON WINDOWS BASH USERS

It turns out that a fair number of us use bash on Windows (through MINGW or bundled with Git, plz – Cygwin is still a bridge too far, for both npm and Node.js). @jakub-g did us all a favor and relaxed the check for npm completion to support MINGW bash. Thanks, Jakub!

THE ONGOING SAGA OF BUNDLED DEPENDENCIES

npm@3.5.0 fixed up a serious issue with how npm@3.4.1 (and potentially npm@3.4.0 and npm@3.3.12) handled the case in which dependencies bundled into a package tarball are handled improperly when one or more of their own dependencies are older than what's latest on the registry. Unfortunately, in fixing that (quite severe) regression (see npm@3.5.0's release notes' for details), we introduced a new (small, and fortunately cosmetic) issue where npm superfluously warns you about bundled dependencies being stale. We have now fixed that, and hope that we haven't introduced any other regressions in the process. :D

  • 20824a7 #10501 Only warn about replacing bundled dependencies when actually doing so. (@iarna)

MAKE NODE-GYP A LITTLE BLUER

  • 1d14d88 node-gyp@3.2.0: Support AIX, use which to find Python, updated to a newer version of gyp, and more! (@bnoordhuis)

A BOUNTEOUS THANKSGIVING CORNUCOPIA OF DOC TWEAKS

These are great! Keep them coming! Sorry for letting them pile up so deep, everybody. Also, a belated Thanksgiving to our Canadian friends, and a happy Thanksgiving to all our friends in the USA.

v3.5.0 (2015-11-19):

TEEN ORCS AT THE GATES

This week heralds the general release of the primary npm registry's new support for private packages for organizations. For many potential users, it's the missing piece needed to make it easy for you to move your organization's private work onto npm. And now it's here! The functionality to support it has been in place in the CLI for a while now, thanks to @zkat's hard work.

During our final testing before the release, our ace support team member @snopeks noticed that there had been some drift between the CLI team's implementation and what npm was actually preparing to ship. In the interests of everyone having a smooth experience with this extremely useful new feature, we quickly made a few changes to square up the CLI and the web site experiences.

NON-OPTIONAL INSTALLS, DEFINITELY NON-OPTIONAL

  • 180263b #10465 When a non-optional dep fails, we check to see if it's only required by ONLY optional dependencies. If it is, we make it fail all the deps in that chain (and roll them back). If it isn't then we give an error.

    We do this by walking up through all of our ancestors until we either hit an optional dependency or the top of the tree. If we hit the top, we know to give the error.

    If you installed a module by hand but didn't --save it, your module won't have the top of the tree as an anscestor and so this code was failing to abort the install with an error

    This updates the logic so that hitting the top OR a module that was requested by the user will trigger the error message. (@iarna)

  • b726a0e #9204 Ideally we would like warnings about your install to come AFTER the output from your compile steps or the giant tree of installed modules.

    To that end, we've moved warnings about failed optional deps to the show after your install completes. (@iarna)

OVERRIDING BUNDLING

  • aed71fb #10482 We've been in our bundled modules code a lot lately, and our last go at this introduced a new bug, where if you had a module a that bundled a module b, which in turn required c, and the version of c that got bundled wasn't compatible with b's package.json, we would then install a compatible version of c, but also erase b at the same time.

    This fixes that. It also reworks our bundled module support to be much closer to being in line with how we handle non-bundled modules and we're hopeful this will reduce any future errors around them. The new structure is hopefully much easier to reason about anyway. (@iarna)

A BRIEF NOTE ON NPM'S BACKWARDS COMPATIBILITY

We don't often have much to say about the changes we make to our internal testing and tooling, but I'm going to take this opportunity to reiterate that npm tries hard to maintain compatibility with a wide variety of Node versions. As this change shows, we want to ensure that npm works the same across:

  • Node.js 0.8
  • Node.js 0.10
  • Node.js 0.12
  • the latest io.js release
  • Node.js 4 LTS
  • Node.js 5

Contributors who send us pull requests often notice that it's very rare that our tests pass across all of those versions (ironically, almost entirely due to the packages we use for testing instead of any issues within npm itself). We're currently beginning an effort, lasting the rest of 2015, to clean up our test suite, and not only get it passing on all of the above versions of Node.js, but working solidly on Windows as well. This is a compounding form of technical debt that we're finally paying down, and our hope is that cleaning up the tests will produce a more robust CLI that's a lot easier to write patches for.

0.8 + npm <1.4 COMPATIBLE? SURE WHY NOT

Hey, you found the feature we added!

  • 231c58a #10337 Add two new flags, first --legacy-bundling which installs your dependencies such that if you bundle those dependencies, npm versions prior to 1.4 can still install them. This eliminates all automatic deduping.

    Second, --global-style which will install modules in your node_modules folder with the same layout as global modules. Only your direct dependencies will show in node_modules and everything they depend on will be flattened in their node_modules folders. This obviously will eliminate some deduping. (@iarna)

TYPOS IN THE LICENSE, OH MY

v3.4.1 (2015-11-12):

ASK FOR NOTHING, GET LATEST

When you run npm install foo, you probably expect that you'll get the latest version of foo, whatever that is. And good news! That's what this change makes it do.

We think this is what everyone wants, but if this causes problems for you, we want to know! If it proves problematic for people we will consider reverting it (preferably before this becomes npm@latest).

Previously, when you ran npm install foo we would act as if you typed npm install foo@*. Now, like any range-type specifier, in addition to matching the range, it would also have to be <= the value of the latest dist-tag. Further, it would exclude prerelease versions from the list of versions considered for a match.

This worked as expected most of the time, unless your latest was a prerelease version, in which case that version wouldn't be used, to everyone's surprise. Worse, if all your versions were prerelease versions it would just refuse to install anything. (We fixed that in npm@3.2.2 with e4a38080.)

  • 1e834c2 #10189 npm-package-arg@4.1.0 Change the default version from * to latest. (@zkat)

BUGS

  • bec4a84 #10338 Failed installs could result in more rollback (removal of just installed packages) than we intended. This bug was first introduced by 83975520. (@iarna)
  • 06c732f #10338 Updating a module could result in the module stealing some of its dependencies from the top level, potentially breaking other modules or resulting in many redundant installations. This bug was first introduced by 971fd47a. (@iarna)
  • 5653366 #9980 npm, when removing a module, would refuse to remove the symlinked binaries if the module itself was symlinked as well. npm goes to some effort to ensure that it doesn't remove things that aren't is, and this code was being too conservative. This code has been rewritten to be easier to follow and to be unit-testable. (@iarna)

LICENSE CLARIFICATION

CLOSER TO GREEN TRAVIS

  • fc12da9 #10232 nock@1.9.0 Downgrade nock to a version that doesn't depend on streams2 in core so that more of our tests can pass in 0.8. (@iarna)

v3.4.0 (2015-11-05):

A NEW FEATURE

This was a group effort, with @isaacs dropping the implementation in back in August. Then, a few days ago, @ashleygwilliams wrote up docs and just today @othiym23 wrote a test.

It's a handy shortcut to update a dependency and then make sure tests still pass.

This new command:

npm install-test x

is the equivalent of running:

npm install x && npm test

BUG FIXES VIA DEPENDENCY UPDATES

  • 31c0080 #8640 npm/normalize-package-data#69 normalize-package-data@2.3.5: Fix a bug where if you didn't specify the name of a scoped module's binary, it would install it such that it was impossible to call it. (@iarna)
  • 02b37bc npm/fstream-npm#14 fstream-npm@1.0.7: Only filter config.gypi when it's in the build directory. (@mscdex)
  • accb9d2 npm/fstream-npm#15 fstream-npm@1.0.6: Stop including directories that happened to have names matching whitelisted npm files in npm module tarballs. The most common cause was that if you had a README directory then everything in it would be included if wanted it or not. (@taion)

DOCUMENTATION FIXES

DEPENDENCY UPDATES FOR THEIR OWN SAKE

v3.3.12 (2015-11-02):

Hi, a little hot-fix release for a bug introduced in 3.3.11. The ENOENT fix last week (f0e2088) broke upgrades of modules that have bundled dependencies (like npm, augh!)

  • aedf7cf #10192 If a bundled module is going to be replacing a module that's currently on disk (for instance, when you upgrade a module that includes bundled dependencies) we want to select the version from the bundle in preference over the one that was there previously. (@iarna)

v3.3.11 (2015-10-29):

This is a dependency update week, so that means no PRs from our lovely users. Look for those next week. As it happens, the dependencies updated were just devdeps, so nothing for you all to worry about.

But the bug fixes, oh geez, I tracked down some really long standing stuff this week!! The headliner is those intermittent ENOENT errors that no one could reproduce consistently? I think they're nailed! But also pretty important, the bug where hapi would install w/ a dep missing? Squashed!

EEEEEEENOENT

  • f0e2088 #10026 Eliminate some, if not many, of the ENOENT errors npm@3 has seen over the past few months. This was happening when npm would, in its own mind, correct a bundled dependency, due to a package.json specifying an incompatible version. Then, when npm extracted the bundled version, what was on disk didn't match its mind and… well, when it tried to act on what was in its mind, we got an ENOENT because it didn't actually exist on disk. (@iarna)

PARTIAL SHRINKWRAPS, NO LONGER A BAD DAY

  • 712fd9c #10153 Imagine that you have a module, let's call it fun-time, and it depends on two dependencies, need-fun@1 and need-time. Further, need-time requires need-fun@2. So after install the logical tree will look like this:

    fun-time
├── need-fun@1
└── need-time
    └── need-fun@2

    Now, the fun-time author also distributes a shrinkwrap, but it only includes the need-fun@1 in it.

    Resolving dependencies would look something like this:

    1. Require need-fun@1: Use version from shrinkwrap (ignoring version)
    2. Require need-time: User version in package.json
    3. Require need-fun@2: Use version from shrinkwrap, which oh hey, is already installed at the top level, so no further action is needed.

    Which results in this tree:

    fun-time
├── need-fun@1
└── need-time

    We're ignoring the version check on things specified in the shrinkwrap so that you can override the version that will be installed. This is because you may want to use a different version than is specified by your dependencies' dependencies' package.json files.

    To fix this, we now only allow overrides of a dependency version when that dependency is a child (in the tree) of the thing that requires it. This means that when we're looking for need-fun@2 we'll see need-fun@1 and reject it because, although it's from a shrinkwrap, it's parent is fun-time and the package doing the requiring is need-time.

    (@iarna)

STRING package.bin AND NON-NPMJS REGISTRIES

  • 3de1463 #9187 If you were using a module with the bin field in your package.json set to a string on a non-npmjs registry then npm would crash, due to the our expectation that the bin field would be an object. We now pass all package.json data through a routine that normalizes the format, including the bin field. (This is the same routine that your package.json is passed through when read off of disk or sent to the registry for publication.) Doing this also ensures that older modules on npm's own registry will be treated exactly the same as new ones. (In the past we weren't always super careful about scrubbing package.json data on publish. And even when we were, those rules have subtly changed over time.) (@iarna)

v3.3.10 (2015-10-22):

Hey you all! Welcome to a busy bug fix and PR week. We've got changes to how npm install replaces dependencies during updates, improvements to shrinkwrap behavior, and all sorts of doc updates.

In other news, npm@3 landed in node master in preparation for node@5 with 41923c0.

UPDATED DEPS NOW MAKE MORE SENSE

  • 971fd47 #9929 Make the tree more consistent by doing updates in place. This means that trees after a dependency version update will more often look the same as after a fresh install. (@iarna)

SHRINKWRAP + DEV DEPS NOW RESPECTED

  • eb28a8c #9647 If a shrinkwrap already has dev deps, don't throw them away when someone later runs npm install --save. (@iarna)

FANTASTIC DOCUMENTATION UPDATES

NEW STANDARD HAS ALWAYS BEEN STANDARD

  • 40c1b0f #9954 Update to standard@5 and reformat the source to work with it. (@cbas)

v3.3.9 (2015-10-15):

This week sees a few small changes ready to land:

TRAVIS NODE 0.8 BUILDS REJOICE

  • 25a234b #9668 Install npm@3's bundled dependencies with npm@2, so that the ancient npm that ships with node 0.8 can install npm@3 directly. (@othiym23)

SMALL ERROR MESSAGE IMPROVEMENT

  • a332f61 #9927 Update error messages where we report a list of versions that you could have installed to show this as a comma separated list instead of as JSON. (@iarna)

DEPENDENCY UPDATES

v3.3.8 (2015-10-12):

This is a small update release, we're reverting 22a3af0 from last week's release, as it is resulting in crashes. We'll revisit this PR during this week.

  • ddde1d5 Revert "lifecycle: Swap out custom logic with add-to-path module" (@iarna)

v3.3.7 (2015-10-08):

So, as Kat mentioned in last week's 2.x release, we're now swapping weeks between accepting PRs and doing dependency updates, in an effort to keep release management work from taking over our lives. This week is a PR week, so we've got a bunch of goodies for you.

Relatedly, this week means 3.3.6 is now latest and it is WAY faster than previous 3.x releases. Give it or this a look!

OPTIONAL DEPS, MORE OPTIONAL

  • 2289234 #9643 #9664 npm@3 was triggering npm@2's build mechanics when it was linking bin files into the tree. This was originally intended to trigger rebuilds of bundled modules, but npm@3's flat module structure confused it. This caused two seemingly unrelated issues. First, failing optional dependencies could under some circumstances (if they were built during this phase) trigger a full build failure. And second, rebuilds were being triggered of already installed modules, again, in some circumstances. Both of these are fixed by disabling the npm@2 mechanics and adding a special rebuild phase for the initial installation of bundled modules. (@iarna)

BAD NAME, NO CRASH

  • b78fec9 #9766 Refactor all attempts to read the module name or package name to go via a single function, with appropriate guards unusual circumstances where they aren't where we expect them. This ultimately will ensure we don't see any more recurrences of the localeCompare error and related crashers. (@iarna)

MISCELLANEOUS BUG FIXES

  • 22a3af0 #9553 Factor the lifecycle code to manage paths out into its own module and use that. (@kentcdodds)
  • 6a29fe3 #9677 Start testing our stuff in node 4 on travis (@fscherwi)
  • 508c6a4 #9669 Make recalculateMetadata more resilient to unexpectedly bogus dependency specifiers. (@tmct)
  • 3c44763 #9643 Update install --only to ignore the NODE_ENV var and just use the only value, if specified. (@watilde)
  • 87336c3 #9879 npm@3's shrinkwrap was refusing to shrinkwrap if an optional dependency was missing– patch it to allow this. (@mantoni)

DOCUMENTATION UPDATES

v3.3.6 (2015-09-30):

I have the most exciting news for you this week. YOU HAVE NO IDEA. Well, ok, maybe you do if you follow my twitter.

Performance just got 5 bazillion times better (under some circumstances, ymmv, etc). So– my test scenario is our very own website. In npm@2, on my macbook running npm ls takes about 5 seconds. Personally it's more than I'd like, but it's entire workable. In npm@3 it has been taking 50 seconds, which is appalling. But after doing some work on Monday isolating the performance issues I've been able to reduce npm@3's run time back down to 5 seconds.

Other scenarios were even worse, there was one that until now in npm@3 that took almost 6 minutes, and has been reduced to 14 seconds.

  • 7bc0d4c cf42217 #8826 Stop using deepclone on super big datastructures. Avoid cloning all-together even when that means mutating things, when possible. Otherwise use a custom written tree-copying function that understands the underlying datastructure well enough to only copy what we absolutely need to. (@iarna)

In other news, look for us this Friday and Saturday at the amazing Open Source and Feelings conference, where something like a third of the company will be attending.

And finally a dependency update

And some subdep updates

v3.3.5 (2015-09-24):

Some of you all may not be aware, but npm is ALSO a company. I tell you this 'cause npm-the-company had an all-staff get together this week, flying in our remote folks from around the world. That was great, but it also basically eliminated normal work on Monday and Tuesday.

Still, we've got a couple of really important bug fixes this week. Plus a lil bit from the now LTS 2.x branch.

ATTENTION WINDOWS USERS

If you previously updated to npm 3 and you try to update again, you may get an error messaging telling you that npm won't install npm into itself. Until you are at 3.3.5 or greater, you can get around this with npm install -f -g npm.

  • bef06f5 #9741 Uh... so... er... it seems that since npm@3.2.0 on Windows with a default configuration, it's been impossible to update npm. Well, that's not actually true, there's a work around (see above), but it shouldn't be complaining in the first place. (@iarna)

STACK OVERFLOWS ON PUBLISH

  • 330b496 #9667 We were keeping track of metadata about your project while packing the tree in a way that resulted in this data being written to packed tar files headers. When this metadata included cycles, it resulted in the the tar file entering an infinite recursive loop and eventually crashing with a stack overflow.

    I've patched this by keeping track of your metadata by closing over the variables in question instead, and I've further restricted gathering and tracking the metadata to times when it's actually needed. (Which is only if you need bundled modules.) (@iarna)

LESS CRASHY ERROR MESSAGES ON BAD PACKAGES

  • 829921f #9741 Packages with invalid names or versions were crashing the installer. These are now captured and warned as was originally intended. (@iarna)

ONE DEPENDENCY UPDATE

AND ONE SUBDEPENDENCY

v3.3.4 (2015-09-17):

This is a relatively quiet release, bringing a few bug fixes and some module updates, plus via the 2.14.5 release some forward compatibility fixes with versions of Node that aren't yet released.

NO BETA NOTICE THIS TIME!!

But, EXCITING NEWS FRIENDS, this week marks the exit of npm@3 from beta. This means that the week of this release, v3.3.3 will become latest and this version (v3.3.4) will become next!!

CRUFT FOR THE CRUFT GODS

What I call "cruft", by which I mean, files sitting around in your node_modules folder, will no longer produce warnings in npm ls nor during npm install. This brings npm@3's behavior in line with npm@2.

BETTER ERROR MESSAGE

MODULE UPDATES

SUB DEP MODULE UPDATES

v3.3.3 (2015-09-10):

This short week brought us brings us a few small bug fixes, a doc change and a whole lotta dependency updates.

Plus, as usual, this includes a forward port of everything in npm@2.14.4.

BETA BUT NOT FOREVER

THIS IS BETA SOFTWARE. npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@v3.x-next and npm@v3.x-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

REMOVE INSTALLED BINARIES ON WINDOWS

So waaaay back at the start of August, I fixed a bug with #9198. That fix made it so that if you had two modules installed that both installed the same binary (eg gulp & gulp-cli), that removing one wouldn't remove the binary if it was owned by the other.

It did this by doing some hocus-pocus that, turns out, was Unix-specific, so on Windows it just threw up its hands and stopped removing installed binaries at all. Not great.

So today we're fixing that– it let us maintain the same safety that we added in #9198, but ALSO works with Windows.

API DOCUMENTATION HAS BEEN SACRIFICED THE API GOD

The documentation of the internal APIs of npm is going away, because it would lead people into thinking they should integrate with npm by using it. Please don't do that! In the future, we'd like to give you a suite of stand alone modules that provide better, more stand alone APIs for your applications to build on. But for now, call the npm binary with process.exec or process.spawn instead.

ALLOW npm link ON WINDOWS W/ PRERELEASE VERSIONS OF NODE

We never meant to have this be a restriction in the first place and it was only just discovered with the recent node 4.0.0 release candidate.

graceful-fs update

We're updating all of npm's deps to use the most recent graceful-fs. This turns out to be important for future not yet released versions of node, because older versions monkey-patch fs in ways that will break in the future. Plus it ALSO makes use of process.binding which is an internal API that npm definitely shouldn't have been using. We're not done yet, but this is the bulk of them.

DEPENDENCY UPDATES

THE DEPENDENCIES OF OUR DEPENDENCIES ARE OUR DEPENDENCIES UPDATES

v3.3.2 (2015-09-04):

PLEASE HOLD FOR THE NEXT AVAILABLE MAINTAINER

This is a tiny little maintenance release, both to update dependencies and to keep npm@3 up to date with changes made to npm@2. @othiym23 is putting out this release (again) as his esteemed colleague @iarna finishes relocating herself, her family, and her sizable anime collection all the way across North America. It contains all the goodies in npm@2.14.3 and one other dependency update.

BETA WARNINGS FOR FUN AND PROFIT

THIS IS BETA SOFTWARE. npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@v3.x-next and npm@v3.x-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

That said, it's getting there! It will be leaving beta very soon!

ONE OTHER DEPENDENCY UPDATE

  • bb5de34 is-my-json-valid@2.12.2: Upgrade to a new, modernized version of json-pointer. (@mafintosh)

v3.3.1 (2015-08-27):

Hi all, this npm@3 update brings you another round of bug fixes. The headliner here is that npm update works again. We're running down the clock on blocker 3.x issues! Shortly after that hits zero we'll be promoting 3.x to latest!!

And of course, we have changes that were brought forward from 2.x. Check out the release notes for 2.14.1 and 2.14.2.

BETA WARNINGS FOR FUN AND PROFIT

THIS IS BETA SOFTWARE. npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@v3.x-next and npm@v3.x-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

NPM UPDATE, NOW AGAIN YOUR FRIEND

  • f130a00 #9095 npm update once again works! Previously, after selecting packages to update, it would then pick the wrong location to run the install from. (@iarna)

MORE VERBOSING FOR YOUR VERBOSE LIFECYCLES

  • d088b7d #9227 Add some additional logging at the verbose and silly levels when running lifecycle scripts. Hopefully this will make debugging issues with them a bit easier! (@saper)

AND SOME OTHER BUG FIXES…

  • f4a5784 #9308 Make fetching metadata for local modules faster! This ALSO means that doing things like running npm repo won't build your module and maybe run prepublish. (@iarna)

  • 4468c92 #9205 Fix a bug where local modules would sometimes not resolve relative links using the correct base path. (@iarna)

  • d395a6b #8995 Certain combinations of packages could result in different install orders for their initial installation than for reinstalls run on the same folder. (@iarna)

  • d119ea6 #9113 Make extraneous packages always up in npm ls. Previously, if an extraneous package had a dependency that depended back on the original package this would result in the package not showing up in ls. (@iarna)

  • 02420dc #9113 Stop warning about missing top level package.json files. Errors in said files will still be reported. (@iarna)

SOME DEP UPDATES

SOME DEPS OF DEPS UPDATES

v3.3.0 (2015-08-13):

This is a pretty EXCITING week. But I may be a little excitable– or possibly sleep deprived, it's sometimes hard to tell them apart. =D So Kat really went the extra mile this week and got the client side support for teams and orgs out in this week's 2.x release. You can't use that just yet, 'cause we have to turn on some server side stuff too, but this way it'll be there for you all the moment we do! Check out the details over in the 2.14.0 release notes!

But we over here in 3.x ALSO got a new feature this week, check out the new --only and --also flags for better control over when dev and production dependencies are used by various npm commands.

That, and some important bug fixes round out this week. Enjoy everyone!

NEVER SHALL NOT BETA THE BETA

THIS IS BETA SOFTWARE. EXCITING NEW BETA WARNING!!! Ok, I fibbed, EXACTLY THE SAME BETA WARNINGS: npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@v3.x-next and npm@v3.x-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

ONLY ALSO DEV

Hey we've got a SUPER cool new feature for you all, thanks to the fantastic work of @davglass and @bengl we have --only=prod, --only=dev, --also=prod and --also=dev options. These apply in various ways to: npm install, npm ls, npm outdated and npm update.

So for instance:

npm install --only=dev

Only installs dev dependencies. By contrast:

npm install --only=prod

Will only install prod dependencies and is very similar to --production but differs in that it doesn't set the environment variables that --production does.

The related new flag, --also is most useful with things like:

npm shrinkwrap --also=dev

As shrinkwraps don't include dev deps by default. This replaces passing in --dev in that scenario.

And that leads into the fact that this deprecates --dev as its semantics across commands were inconsistent and confusing.

DON'T TOUCH! THAT'S NOT YOUR BIN

  • b31812e #8996 When removing a module that has bin files, if one that we're going to remove is a symlink to a DIFFERENT module, leave it alone. This only happens when you have two modules that try to provide the same bin. (@iarna)

THERE'S AN END IN SIGHT

  • d2178a9 #9223 Close a bunch of infinite loops that could show up with symlink cycles in your dependencies. (@iarna)

OOPS DIDN'T MEAN TO FIX THAT

Well, not just yet. This was scheduled for next week, but it snuck into 2.x this week.

  • 139dd92 #8716 npm init will now only pick up the modules you install, not everything else that got flattened with them. (@iarna)

v3.2.2 (2015-08-08):

Lot's of lovely bug fixes for npm@3. I'm also suuuuper excited that I think we have a handle on stack explosions that effect a small portion of our users. We also have some tantalizing clues as to where some low hanging fruit may be for performance issues.

And of course, in addition to the npm@3 specific bug fixes, there are some great one's coming in from npm@2! @othiym23 put together that release this week– check out its release notes for the deets.

AS ALWAYS STILL BETA

THIS IS BETA SOFTWARE. Just like the airline safety announcements, we're not taking this plane off till we finish telling you: npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@v3.x-next and npm@v3.x-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

BUG FIXES

  • a8c8a13 #9050 Resolve peer deps relative to the parent of the requirer (@iarna)
  • 05f0226 #9077 Fix crash when saving git+ssh urls (@iarna)
  • e4a3808 #8951 Extend our patch to allow * to match something when a package only has prerelease versions to everything and not just the cache. (@iarna)
  • d135abf #8871 Don't warn about a missing package.json or missing fields in the global install directory. (@iarna)

DEP VERSION BUMPS

v3.2.1 (2015-07-31):

AN EXTRA QUIET RELEASE

A bunch of stuff got deferred for various reasons, which just means more branches to land next week!

Don't forget to check out Kat's 2.x release for other quiet goodies.

AS ALWAYS STILL BETA

THIS IS BETA SOFTWARE. Yes, we're still reminding you of this. No, you can't be excused. npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@v3.x-next and npm@v3.x-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

MAKING OUR TESTS TEST THE THING THEY TEST

  • 6e53c3d #8985 Many thanks to @bengl for noticing that one of our tests wasn't testing what it claimed it was testing! (@bengl)

MY PACKAGE.JSON WAS ALREADY IN THE RIGHT ORDER

  • eb2c7aa #9068 Stop sorting keys in the package.json that we haven't edited. Many thanks to @Qix- for bringing this up and providing a first pass at a patch for this. (@iarna)

DEV DEP UPDATE

v3.2.0 (2015-07-24):

MORE CONFIG, BETTER WINDOWS AND A BUG FIX

This is a smallish release with a new config option and some bug fixes. And lots of module updates.

BETA BETAS ON

THIS IS BETA SOFTWARE. Yes, we're still reminding you of this. No, you can't be excused. npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@v3.x-next and npm@v3.x-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

NEW CONFIGS, LESS PROGRESS

  • 423d8f7 #8704 Add the ability to disable the new progress bar with --no-progress (@iarna)

AND BUG FIXES

  • b3ee452 #9038 We previously disabled the use of the new fs.access API on Windows, but the bug we were seeing is fixed in io.js@1.5.0 so we now use fs.access if you're using that version or greater. (@iarna)

  • b181fa3 #8921 #8637 Rejigger how we validate modules for install. This allow is to fix a problem where arch/os checking wasn't being done at all. It also made it easy to add back in a check that declines to install a module in itself unless you force it. (@iarna)

AND A WHOLE BUNCH OF SUBDEP VERSIONS

These are all development dependencies and semver-compatible subdep upgrades, so they should not have visible impact on users.

MERGED FORWARD

  • As usual, we've ported all the npm@2 goodies in this week's v2.13.3 release.

v3.1.3 (2015-07-17):

Rebecca: So Kat, I hear this week's other release uses a dialog between us to explain what changed?

Kat: Well, you could say that…

Rebecca: I would! This week I fixed more npm@3 bugs!

Kat: That sounds familiar.

Rebecca: Eheheheh, well, before we look at those, a word from our sponsor…

BETA IS AS BETA DOES

THIS IS BETA SOFTWARE. Yes, we're still reminding you of this. No, you can't be excused. npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@v3.x-next and npm@v3.x-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

Rebecca: Ok, enough of the dialoguing, that's Kat's schtick. But do remember kids, betas hide in dark hallways waiting to break your stuff, stuff like…

SO MANY LINKS YOU COULD MAKE A CHAIN

  • 6d69ec9 #8967 Removing a module linked into your globals would result in having all of its subdeps removed. Since the npm release process does exactly this, it burned me -every- -single- -week-. =D While we're here, we also removed extraneous warns that used to spill out when you'd remove a symlink. (@iarna)

  • fdb360f #8874 Linking scoped modules was failing outright, but this fixes that and updates our tests so we don't do it again. (@iarna)

WE'LL TRY NOT TO CRACK YOUR WINDOWS

  • 9fafb18 #8701 npm@3 introduced permissions checks that run before it actually tries to do something. This saves you from having an install fail half way through. We did this using the shiny new fs.access function available in node 0.12 and io.js, with fallback options for older nodes. Unfortunately the way we implemented the fallback caused racey problems for Windows systems. This fixes that by ensuring we only ever run any one check on a directory once. BUT it turns out there are bugs in fs.access on Windows. So this ALSO just disables the use of fs.access on Windows entirely until that settles out. (@iarna)

ZOOM ZOOM, DEP UPDATES

  • 5656baa gauge@1.2.2: Better handle terminal resizes while printing the progress bar (@iarna)

MERGED FORWARD

v3.1.2

SO VERY BETA RELEASE

So, v3.1.1 managed to actually break installing local modules. And then immediately after I drove to an island for the weekend. 😁 So let's get this fixed outside the usual release train!

Fortunately it didn't break installing global modules and so you could swap it out for another version at least.

DISCLAIMER MEANS WHAT IT SAYS

THIS IS BETA SOFTWARE. Yes, we're still reminding you of this. No, you can't be excused. npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@v3.x-next and npm@v3.x-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

THIS IS IT, THE REASON

v3.1.1

RED EYE RELEASE

Rebecca's up too late writing tests, so you can have npm@3 bug fixes! Lots of great new issues from you all! ❤️️ Keep it up!

YUP STILL BETA, PLEASE PAY ATTENTION

THIS IS BETA SOFTWARE. Yes, we're still reminding you of this. No, you can't be excused. npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@v3.x-next and npm@v3.x-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

BOOGS

  • 9badfd6 #8608 Make global installs and uninstalls MUCH faster by only reading the directories of modules referred to by arguments. (@iarna
  • 075a5f0 #8660 Failed optional deps would still result in the optional deps own dependencies being installed. We now find them and fail them out of the tree. (@iarna
  • c9fbbb5 #8863 The "no compatible version found" error message was including only the version requested, not the name of the package we wanted. Ooops! (@iarna
  • 32e6bbd #8806 The "uninstall" lifecycle was being run after all of a module's dependencies has been removed. This reverses that order-- this means "uninstall" lifecycles can make use of the package's dependencies. (@iarna

MERGED FORWARD

v3.1.0 (2015-07-02):

This has been a brief week of bug fixes, plus some fun stuff merged forward from this weeks 2.x release. See the 2.13.0 release notes for details on that.

You all have been AWESOME with all the npm@3 bug reports! Thank you and keep up the great work!

NEW PLACE, SAME CODE

Remember how last week we said npm@3 would go to 3.0-next and latest tags? Yeaaah, no, please use npm@v3.x-next and npm@v3.x-latest going forward.

I dunno why we said "suuure, we'll never do a feature release till we're out of beta" when we're still forward porting npm@2.x features. ¯\_(ツ)_/¯

If you do accidentally use the old tag names, I'll be maintaining them for a few releases, but they won't be around forever.

YUP STILL BETA, PLEASE PAY ATTENTION

THIS IS BETA SOFTWARE. npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@v3.x-next and npm@v3.x-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

BUGS ON THE WINDOWS

  • 0030ade #8685 Windows would hang when trying to clone git repos (@euprogramador)
  • b259bcc #8786 Windows permissions checks would cause installations to fail under some circumstances. We're disabling the checks entirely for this release. I'm hoping to check back with this next week to get a Windows friendly fix in. (@iarna)

SO MANY BUGS SQUASHED, JUST CALL US RAID

  • 0848698 #8686 Stop leaving progress bar cruft on the screen during publication (@ajcrites)
  • 57c3cea #8695 Remote packages with shrinkwraps made npm cause node + iojs to explode and catch fire. NO MORE. (@iarna)
  • 2875ba3 #8723 I uh, told you that engineStrict checking had gone away last week. TURNS OUT I LIED. So this is making that actually be true. (@iarna)
  • 28064e5 #3358 Consistently allow Unicode BOMs at the start of package.json files. Previously this was allowed some of time, like when you were installing modules, but not others, like running npm version or installing w/ --save. (@iarna)
  • 3cb6ad2 #8736 npm@3 wasn't running the "install" lifecycle in your current (toplevel) module. This broke modules that relied on C compilation. BOO. (@iarna)
  • 68da583 #8766 To my great shame, npm link package wasn't working AT ALL if you didn't have package already installed. (@iarna)
  • edd7448 read-package-tree@5.0.0: This update makes read-package-tree not explode when there's bad data in your node_modules folder. npm@2 silently ignores this sort of thing. (@iarna)
  • 0bb08c8 #8778 RELATEDLY, we now show any errors from your node_modules folder after your installation completes as warnings. We're also reporting these in npm ls now. (@iarna)
  • 6c248ff #8779 Hey, you know how we used to complain if your package.json was missing stuff? Well guess what, we are again. I know, I know, you can thank me later. (@iarna)
  • d6f7c98 So, when we were rolling back after errors we had untested code that tried to undo moves. Being untested it turns out it was very broken. I've removed it until we have time to do this right. (@iarna)

NEW VERSION

Just the one. Others came in via the 2.x release. Do check out its changelog, immediately following this message.

v3.0.0 (2015-06-25):

Wow, it's finally here! This has been a long time coming. We are all delighted and proud to be getting this out into the world, and are looking forward to working with the npm user community to get it production-ready as quickly as possible.

npm@3 constitutes a nearly complete rewrite of npm's installer to be easier to maintain, and to bring a bunch of valuable new features and design improvements to you all.

@othiym23 and @isaacs have been talking about the changes in this release for well over a year, and it's been the primary focus of @iarna since she joined the team.

Given that this is a near-total rewrite, all changes listed here are @iarna's work unless otherwise specified.

NO, REALLY, READ THIS PARAGRAPH. IT'S THE IMPORTANT ONE.

THIS IS BETA SOFTWARE. npm@3 will remain in beta until we're confident that it's stable and have assessed the effect of the breaking changes on the community. During that time we will still be doing npm@2 releases, with npm@2 tagged as latest and next. We'll also be publishing new releases of npm@3 as npm@3.0-next and npm@3.0-latest alongside those versions until we're ready to switch everyone over to npm@3. We need your help to find and fix its remaining bugs. It's a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don't use it for production maintenance or frontline continuous deployment just yet.

BREAKING CHANGES

peerDependencies

grunt, gulp, and broccoli plugin maintainers take note! You will be affected by this change!

  • #6930 (#6565) peerDependencies no longer cause anything to be implicitly installed. Instead, npm will now warn if a packages peerDependencies are missing, but it's up to the consumer of the module (i.e. you) to ensure the peers get installed / are included in package.json as direct dependencies or devDependencies of your package.
  • #3803 npm also no longer checks peerDependencies until after it has fully resolved the tree.

This shifts the responsibility for fulfilling peer dependencies from library / framework / plugin maintainers to application authors, and is intended to get users out of the dependency hell caused by conflicting peerDependency constraints. npm's job is to keep you out of dependency hell, not put you in it.

engineStrict
  • #6931 The rarely-used package.json option engineStrict has been deprecated for several months, producing warnings when it was used. Starting with npm@3, the value of the field is ignored, and engine violations will only produce warnings. If you, as a user, want strict engines field enforcement, just run npm config set engine-strict true.

As with the peer dependencies change, this is about shifting control from module authors to application authors. It turns out engineStrict was very difficult to understand even harder to use correctly, and more often than not just made modules using it difficult to deploy.

npm view
  • 77f1aec With npm view (aka npm info), always return arrays for versions, maintainers, etc. Previously npm would return a plain value if there was only one, and multiple values if there were more. (@KenanY)

KNOWN BUGS

Again, this is a BETA RELEASE, so not everything is working just yet. Here are the issues that we already know about. If you run into something that isn't on this list, let us know!

  • #8575 Circular deps will never be removed by the prune-on-uninstall code.
  • #8588 Local deps where the dep name and the name in the package.json differ don't result in an error.
  • #8637 Modules can install themselves as direct dependencies. npm@2 declined to do this.
  • #8660 Dependencies of failed optional dependencies aren't rolled back when the optional dependency is, and then are reported as extraneous thereafter.

NEW FEATURES

The multi-stage installer!

  • #5919 Previously the installer had a set of steps it executed for each package and it would immediately start executing them as soon as it decided to act on a package.

    But now it executes each of those steps at the same time for all packages, waiting for all of one stage to complete before moving on. This eliminates many race conditions and makes the code easier to reason about.

This fixes, for instance:

  • #6926 (#5001, #6170) install and postinstall lifecycle scripts now only execute after all the module with the script's dependencies are installed.
Install: it looks different!

You'll now get a tree much like the one produced by npm ls that highlights in orange the packages that were installed. Similarly, any removed packages will have their names prefixed by a -.

Also, npm outdated used to include the name of the module in the Location field:

Package                Current  Wanted  Latest  Location
deep-equal             MISSING   1.0.0   1.0.0  deep-equal
glob                     4.5.3   4.5.3  5.0.10  rimraf > glob

Now it shows the module that required it as the final point in the Location field:

Package                Current  Wanted  Latest  Location
deep-equal             MISSING   1.0.0   1.0.0  npm
glob                     4.5.3   4.5.3  5.0.10  npm > rimraf

Previously the Location field was telling you where the module was on disk. Now it tells you what requires the module. When more than one thing requires the module you'll see it listed once for each thing requiring it.

Install: it works different!
  • #6928 (#2931 #2950) npm install when you have an npm-shrinkwrap.json will ensure you have the modules specified in it are installed in exactly the shape specified no matter what you had when you started.
  • #6913 (#1341 #3124 #4956 #6349 #5465) npm install when some of your dependencies are missing sub-dependencies will result in those sub-dependencies being installed. That is, npm install now knows how to fix broken installs, most of the time.
  • #5465 If you directly npm install a module that's already a subdep of something else and your new version is incompatible, it will now install the previous version nested in the things that need it.
  • a2b50cf #5693 When installing a new module, if it's mentioned in your npm-shrinkwrap.json or your package.json use the version specifier from there if you didn't specify one yourself.
Flat, flat, flat!

Your dependencies will now be installed maximally flat. Insofar as is possible, all of your dependencies, and their dependencies, and THEIR dependencies will be installed in your project's node_modules folder with no nesting. You'll only see modules nested underneath one another when two (or more) modules have conflicting dependencies.

  • #3697 This will hopefully eliminate most cases where Windows users ended up with paths that were too long for Explorer and other standard tools to deal with.
  • #6912 (#4761 #4037) This also means that your installs will be deduped from the start.
  • #5827 This deduping even extends to git deps.
  • #6936 (#5698) Various commands are dedupe aware now.

This has some implications for the behavior of other commands:

  • npm uninstall removes any dependencies of the module that you specified that aren't required by any other module. Previously, it would only remove those that happened to be installed under it, resulting in left over cruft if you'd ever deduped.
  • npm ls now shows you your dependency tree organized around what requires what, rather than where those modules are on disk.
  • #6937 npm dedupe now flattens the tree in addition to deduping.

And bundling of dependencies when packing or publishing changes too:

  • #2442 bundledDependencies no longer requires that you specify deduped sub deps. npm can now see that a dependency is required by something bundled and automatically include it. To put that another way, bundledDependencies should ONLY include things that you included in dependencies, optionalDependencies or devDependencies.
  • #5437 When bundling a dependency that's both a devDependency and the child of a regular dependency, npm bundles the child dependency.

As a demonstration of our confidence in our own work, npm's own dependencies are now flattened, deduped, and bundled in the npm@3 style. This means that npm@3 can't be packed or published by npm@2, which is something to be aware of if you're hacking on npm.

Shrinkwraps: they are a-changin'!

First of all, they should be idempotent now (#5779). No more differences because the first time you install (without npm-shrinkwrap.json) and the second time (with npm-shrinkwrap.json).

  • #6781 Second, if you save your changes to package.json and you have npm-shrinkwrap.json, then it will be updated as well. This applies to all of the commands that update your tree:
    • npm install --save
    • npm update --save
    • npm dedupe --save (#6410)
    • npm uninstall --save
  • #4944 (#5161 #5448) Third, because node_modules folders are now deduped and flat, shrinkwrap has to also be smart enough to handle this.

And finally, enjoy this shrinkwrap bug fix:

  • #3675 When shrinkwrapping a dependency that's both a devDependency and the child of a regular dependency, npm now correctly includes the child.
The Age of Progress (Bars)!
  • #6911 (#1257 #5340 #6420) The spinner is gone (yay? boo? will you miss it?), and in its place npm has progress bars, so you actually have some sense of how long installs will take. It's provided in Unicode and non-Unicode variants, and Unicode support is automatically detected from your environment.

TINY JEWELS

The bottom is where we usually hide the less interesting bits of each release, but each of these are small but incredibly useful bits of this release, and very much worth checking out:

  • 9ebe312 Build system maintainers, rejoice: npm does a better job of cleaning up after itself in your temporary folder.
  • #6942 Check for permissions issues prior to actually trying to install anything.
  • Emit warnings at the end of the installation when possible, so that they'll be on your screen when npm stops.
  • #3505 npm --dry-run: You can now ask that npm only report what it would have done with the new --dry-run flag. This can be passed to any of the commands that change your node_modules folder: install, uninstall, update and dedupe.
  • 81b46fb npm now knows the correct URLs for npm bugs and npm repo for repositories hosted on Bitbucket and GitLab, just like it does for GitHub (and GitHub support now extends to projects hosted as gists as well as traditional repositories).
  • 5be4008a npm has been cleaned up to pass the standard style checker. Forrest and Rebecca both feel this makes it easier to read and understand the code, and should also make it easier for new contributors to put merge-ready patches. (@othiym23)

ZARRO BOOGS

  • 6401643 Make sure the global install directory exists before installing to it. (@thefourtheye)
  • #6158 When we remove modules we do so inside-out running unbuild for each one.
  • 960a765 The short usage information for each subcommand has been brought in sync with the documentation. (@smikes)