Keep the basic workflow test (checking what a user would do to get started) in bash, but otherwise it's more convenient for most tests to be in python.
This makes
emsdk install 1.38.33
work (1.38.33 is from the new builders).
Also works with the old notation, sdk-1.38.33-64bit, and also supports -(upstream|fastcomp).
This makes emsdk [install|activate] latest get the latest fastcomp release from emscripten-releases, replacing our usage of the old mozilla build infrastructure.
This is only for "latest", so there should be no change to older releases.
This does remove non-release old stuff from the mozilla infrastructure, like "nightly" builds (that I don't think were even working?). After this PR, only old releases should be used from there.
Context: we've asked that people test the emscripten-releases builds for 1.38.33, and fixed a few issues people found. There are no open showstoppers, so this is us moving to the next step, serving latest builds from the new infrastructure.
We can remove this now that we have emscripten-releases working and use it everywhere we used the emsdk's waterfall integration.
In particular this should fix the current lkgr.json errors people are seeing (by removing all the lkgr stuff), which I believe started when I refactored that code while doing the releases work - I must have gotten something wrong on non-linux OSes. But anyhow, easier to remove that unnecessary code than fix it at this point.
emsdk install tot-upstream will install the very latest build from emscripten-releases. These builds are useful for emscripten github CI.
There isn't a latest or lkgr for emscripten-releases currently. What this does instead is get the git repo, and check if builds exist for the latest commits there, returning the latest of those.
This also makes us not update the emscripten-version.txt file if we are not an actual version. That is, for a tot build we leave that file unchanged in the emscripten checkout.
There is also a tot-fastcomp for fastcomp.
Updated to our current initial plan here:
* Add emscripten-releases-tags.txt which is a JSON file with a map of tag name to git hash in the emscripten-releases repo. Right now this file is just checked in here; in the future we may make the emsdk update it from git tags with update-tags etc.
* That file contains a "latest" tag, which is fetched for when getting latest-releases-[upstream|fastcomp]. We may want that to eventually be updated based on the latest green stuff on the bots perhaps.
* Rename the current sdk-[upstream|fastcomp]-* etc. to sdk-waterfall-[upstream\|fastcomp]-*. Then we'll have sdk-releases-[upstream|fastcomp]-* etc. This is just an internal name change - emsdk install latest-upstream still works as before, and still uses the waterfall (so we don't break our github CI).
* Add support for emsdk install latest-releases-[upstream\|fastcomp]
* Add a test using the actual emscripten-releases builds
While doing so, we:
* keep the latest activation (e.g., the user may have activated latest and then latest-upstream, then the upstream LLVM is what is desired).
* keep the order of keys fixed (so the relative order of lines in the .emscripten file is fixed)
This adds some assertions in the Dockerfile, to verify we have one LLVM_ROOT command, and it is the right one.
Fixes#194
(This was reverted by mistake. It had a bad commit message though, so relanding with a nicer one is nice anyhow.)
With this, we can do emsdk install latest-fastcomp and it installs fastcomp from the waterfall. That is, we then have 3 main sdks people might want to use:
* latest which installs fastcomp-llvm (plus emscripten etc.) from the mozilla infrastructure. (fetches the last emscripten version there)
* latest-upstream which installs upstream-llvm (plus emscripten etc.) from the waterfall infrastructure. (fetches the last known good revision (lkgr) there)
* latest-fastcomp which installs fastcomp-llvm (plus emscripten etc.) from the waterfall infrastructure. (fetches the last known good revision (lkgr) there)
The first and last are currently somewhat overlapping in that both fetch a build of fastcomp. However, as we transition away from the mozilla infrastructure, we could just make latest an alias for latest-fastcomp. (And later, when we're ready to switch to the wasm backend by default, the alias could switch to latest-upstream.)
This makes it possible to tell the emsdk to get "latest-upstream", which fetches the latest lkgr from there. This will probably be a common use pattern, I expect we may want to recommend users start trying out the wasm backend that way soon. This will also let us simplify this code: https://github.com/kripken/emscripten/blob/incoming/.circleci/config.yml#L334
Aside from adding the "latest-upstream" alias, this PR has
* A few minor cleanups in the emsdk code.
( Minor restructuring of how we define the upstream stuff in the manifest: It seemed odd to have 3 things (clang, emscripten, binaryen) that are all coming from a single archive from the waterfall. Simpler to have just one - the archive is one big lump, there's no way to download just part of it.
* Also add node 8.9.1 as a dependency of the upstream sdk, which makes things usable out of the box (node.js is the one thing not provided by the waterfall archive).
