admin/ci-emsdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Added dockerfiles form trzeci/emscripten-docker (#368)

Browse Source 
As a continuation from: https://github.com/emscripten-core/emscripten/issues/8742

This PR provides adopted Dockerfile from https://github.com/trzecieu/emscripten-docker/blob/master/docker/trzeci/emscripten-fastcomp/Dockerfile which supports builds:
  * fastcomp
  * upstream

Features:
* An image can be build with fastcomp and upstream version of Emscripten SDK
* It applies some extra size optimizations (that might be removed in sake of clarity / stability)
* It creates conventional `1000:1000` user `emscripten`
* It creates extra entrypoint that mitigates running as non-root user (Some context: https://trzeci.eu/fixing-permission-of-files-created-from-docker/)
* It has simple-stupid tests performed during building
This commit is contained in:
Piotr Paczkowski (trzeci.eu)
2019-10-11 20:21:47 +02:00
committed by Sam Clegg
parent 7f9af7bd1a
commit c56422590b
5 changed files with 437 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

235
docker/Dockerfile Normal file
View File

@@ -0,0 +1,235 @@
FROM debian:stretch AS stage_build
# ------------------------------------------------------------------------------
# Supports only 1.38.40+, accpets also '-upstream' variants
ARG EMSCRIPTEN_VERSION=1.38.43
ARG EMSDK_CHANGESET=master
# ------------------------------------------------------------------------------
# NOTE: Any change of following variables should be reflected in ./entrypoint file
ENV EMSDK /emsdk
ENV EM_DATA ${EMSDK}/.data
ENV EM_CONFIG ${EMSDK}/.emscripten
ENV EM_CACHE ${EM_DATA}/cache
ENV EM_PORTS ${EM_DATA}/ports
# ------------------------------------------------------------------------------
RUN echo "## Start building" \
\
&& echo "## Update and install packages" \
&& apt-get -qq -y update \
&& apt-get -qq install -y --no-install-recommends \
libxml2 \
wget \
git-core \
ca-certificates \
build-essential \
file \
python python-pip \
python3 python3-pip \
\
&& echo "## Done"
RUN echo "## Get EMSDK" \
&& git clone https://github.com/emscripten-core/emsdk.git ${EMSDK} \
&& cd ${EMSDK} && git reset --hard ${EMSDK_CHANGESET} \
\
&& ./emsdk.py update-tags \
&& echo "## Done"
RUN echo "## Install Emscripten" \
&& cd ${EMSDK} \
&& ./emsdk install ${EMSCRIPTEN_VERSION} \
\
&& echo "## Done"
# This generates configuration that contains all valid paths according to installed SDK
RUN cd ${EMSDK} \
&& echo "## Generate standard configuration" \
\
&& ./emsdk activate ${EMSCRIPTEN_VERSION} --embedded \
&& ./emsdk construct_env > /dev/null \
&& cat ${EMSDK}/emsdk_set_env.sh \
\
# remove wrongly created entry with EM_CACHE, variable will be picked up from ENV
&& sed -i -e "/EM_CACHE/d" ${EMSDK}/emsdk_set_env.sh \
# add a link to tools like asm2wasm in a system path
# asm2wasm (and friends might be places either in ./upstream of ./fastcomp folder, hence detection is needed)
&& printf "export PATH=$(dirname $(find . -name asm2wasm -exec readlink -f {} +)):\$PATH\n" >> ${EMSDK}/emsdk_set_env.sh \
\
&& echo "## Done"
# Create a structure and make mutable folders accessible for r/w
RUN cd ${EMSDK} \
&& echo "## Create .data structure" \
&& for mutable_dir in ${EM_DATA} ${EM_PORTS} ${EM_CACHE} ${EMSDK}/zips ${EMSDK}/tmp; do \
mkdir -p ${mutable_dir}; \
chmod -R 777 ${mutable_dir}; \
done \
\
&& echo "## Done"
# Create an entrypoint that activates Emscripten SDK and helps running this image as non-root user
COPY entrypoint ${EMSDK}/
# Create symbolic links for critical Emscripten Tools
# This is important for letting people using Emscripten in Dockerfiles without activation
# As each Emscripten release is placed to a different folder (i.e. /emsdk/emscripten/tag-1.38.31)
RUN echo "## Create symbolic links" \
&& . ${EMSDK}/emsdk_set_env.sh \
\
&& mkdir -p ${EMSDK}/llvm ${EMSDK}/emscripten ${EMSDK}/binaryen \
\
&& ln -s $(dirname $(which node))/.. ${EMSDK}/node/current \
&& ln -s $(dirname $(which clang))/.. ${EMSDK}/llvm/clang \
&& ln -s $(dirname $(which emcc)) ${EMSDK}/emscripten/sdk \
\
&& ln -s $(dirname $(which asm2wasm)) ${EMSDK}/binaryen/bin \
\
&& echo "## Done"
# Clean up emscripten installation and strip some symbols
RUN echo "## Aggresive optimization: Remove debug symbols" \
&& apt-get -qq -y update && apt-get -qq install -y --no-install-recommends \
binutils \
&& . ${EMSDK}/emsdk_set_env.sh \
# Remove debugging symbols from embedded node (extra 7MB)
&& strip -s `which node` \
# Tests consume ~80MB disc space
&& rm -fr ${EMSDK}/llvm/clang/emscripten/tests \
# strip out symbols from clang (~extra 50MB disc space)
&& find ${EMSDK}/llvm/clang/bin -type f -exec strip -s {} + || true \
&& find ${EMSDK}/llvm/clang/fastcomp/bin -type f -exec strip -s {} + || true \
&& echo "## Done"
# Populate Emscripten SDK cache with libc++, to improve further compilation times.
RUN echo "## Pre-populate cache" \
&& . ${EMSDK}/emsdk_set_env.sh \
\
&& embuilder.py build SYSTEM \
\
&& mkdir -p /tmp/emscripten_test \
&& cd /tmp/emscripten_test \
\
&& printf '#include <iostream>\nint main(){std::cout << "HELLO FROM DOCKER C++"<<std::endl;return 0;}' > test.cpp \
&& em++ --std=c++11 test.cpp -o test.js -s WASM=0 && node test.js \
&& em++ --std=c++11 -g3 test.cpp -o test.js -s WASM=0 && node test.js \
&& em++ --std=c++11 test.cpp -o test.js -s WASM=1 && node test.js \
\
&& cd / \
&& rm -fr /tmp/emscripten_test \
\
# some files were created, and we need to make sure that those can be accessed by non-root people
&& chmod -R 777 ${EM_DATA} \
\
# cleanup
&& find ${EMSDK} -name "*.pyc" -exec rm {} \; \
\
&& echo "## Done"
# ------------------------------------------------------------------------------
# -------------------------------- STAGE DEPLOY --------------------------------
# ------------------------------------------------------------------------------
FROM debian:buster-slim AS stage_deploy
COPY --from=stage_build /emsdk /emsdk
# Fallback in case Emscripten isn't activated.
# This will let use tools offered by this image inside other Docker images (sub-stages) or with custom / no entrypoint
ENV EMSDK /emsdk
ENV EMSCRIPTEN=${EMSDK}/emscripten/sdk
ENV EM_DATA ${EMSDK}/.data
ENV EM_CONFIG ${EMSDK}/.emscripten
ENV EM_CACHE ${EM_DATA}/cache
ENV EM_PORTS ${EM_DATA}/ports
# Fallback in case Emscripten isn't activated
# Expose Major tools to system PATH, so that emcc, node, asm2wasm etc can be used without activation
ENV PATH="${EMSDK}:${EMSDK}/emscripten/sdk:${EMSDK}/llvm/clang/bin:${EMSDK}/node/current/bin:${EMSDK}/binaryen/bin:${PATH}"
# Use entrypoint that's coming from emscripten-slim image. It sets all required system paths and variables
ENTRYPOINT ["/emsdk/entrypoint"]
# ------------------------------------------------------------------------------
# Create a 'standard` 1000:1000 user
# Thanks to that this image can be executed as non-root user and created files will not require root access level on host machine
# Please note that this solution even if widely spread (i.e. Node.js uses it) is far from perfect as user 1000:1000 might not exist on
# host machine, and in this case running any docker image will cause other random problems (mostly due `$HOME` pointing to `/`)
# This extra user works nicely with entrypoint provided in `/emsdk/entrypoint` as it detects case explained before.
RUN echo "## Create emscripten user (1000:1000)" \
&& groupadd --gid 1000 emscripten \
&& useradd --uid 1000 --gid emscripten --shell /bin/bash --create-home emscripten \
\
&& echo "## Done"
# ------------------------------------------------------------------------------
RUN echo "## Update and install packages" \
# mitigate problem with create symlink to man for base debian image
&& mkdir -p /usr/share/man/man1/ \
\
&& apt-get -qq -y update && apt-get -qq install -y --no-install-recommends \
libxml2 \
ca-certificates \
python \
python3 \
python-pip \
python3-pip \
wget \
curl \
zip \
unzip \
git \
ssh-client \
build-essential \
make \
ant \
libidn11 \
cmake \
openjdk-11-jre-headless \
\
# Standard Cleanup on Debian images
&& apt-get -y clean \
&& apt-get -y autoclean \
&& apt-get -y autoremove \
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf /var/cache/debconf/*-old \
&& rm -rf /usr/share/doc/* \
&& rm -rf /usr/share/man/?? \
&& rm -rf /usr/share/man/??_* \
&& echo "## Done"
# ------------------------------------------------------------------------------
# Internal test suite of tools that this image provides
COPY test_dockerimage.sh ${EMSDK}/
RUN echo "## Internal Testing of image (activated)" \
&& . ${EMSDK}/emsdk_set_env.sh \
&& ${EMSDK}/test_dockerimage.sh \
\
&& echo "## Done"
RUN echo "## Internal Testing of image (not-activated)" \
&& ${EMSDK}/test_dockerimage.sh \
\
&& echo "## Done"
# ------------------------------------------------------------------------------
# Copy this Dockerimage into image, so that it will be possible to recreate it later
COPY Dockerfile /emsdk/dockerfiles/emscripten-core/emsdk/
LABEL maintainer="kontakt@trzeci.eu" \
org.label-schema.name="emscripten" \
org.label-schema.description="The official container with Emscripten SDK" \
org.label-schema.url="https://emscripten.org" \
org.label-schema.vcs-url="https://github.com/emscripten-core/emsdk" \
org.label-schema.docker.dockerfile="/docker/Dockerfile"
# ------------------------------------------------------------------------------

21
docker/Makefile Normal file
View File

@@ -0,0 +1,21 @@
#!/usr/bin/env make
# Emscripten version to build: Should match the version that has been already released.
# i.e.: 1.38.45, 1.38.45-upstream
version =
alias =
.TEST:
ifndef version
$(error argument 'version' is not set. Please call `make version=SOME_VERSION ...`)
endif
build: .TEST
docker build --build-arg=EMSCRIPTEN_VERSION=${version} --tag emscripten/emscripten:${version} .
push: .TEST
docker push emscripten/emscripten:${version}
ifdef alias
docker tag emscripten/emscripten:${version} emscripten/emscripten:${alias}
docker push emscripten/emscripten:${alias}
endif

135
docker/README.md Normal file
View File

@@ -0,0 +1,135 @@
# Dockerfile for EMSDK
This Dockerfile builds a self-contained version of emsdk that enables emscripten to be used without any
other installation on the host system.
It is published at https://hub.docker.com/u/emscripten/emscripten
### Usage
Simple usage of this container to compile a hello-world
```bash
# create helloworld.cpp
cat << EOF > helloworld.cpp
#include <iostream>
int main() {
std::cout << "Hello World!" << std::endl;
return 0;
}
EOF
```
```bash
# compile with docker image
docker run \
--rm \
-v $(pwd):$(pwd) \
-u $(id -u):$(id -g) \
emscripten/emscripten \
emcc helloworld.cpp -o helloworld.js
# execute on host machine
node helloworld.js
```
Teardown of compilation command:
|part|description|
|---|---|
|`docker run`| A standard command to run a command in a container|
|`--rm`|remove a container after execution (optimization)|
|`-v $(pwd):$(pwd)`|Mounting current folder from the host system into mirrored path on the container<br>TIP: This helps to investigate possible problem as we preserve exactly the same paths like in host. In such case modern editors (like Sublime, Atom, VS Code) let us to CTRL+Click on a problematic file |
|`-u $(id -u):$(id -g)`| Run the container as a non-root user with the same UID and GID as local user. Hence all files produced by this are accessible to non-root users|
|`emscripten/emscripten`|Get the latest tag of this container|
|`emcc helloworld.cpp -o helloworld.js`|Execute `emcc` command with following arguments inside container, effectively compile our source code|
### Building Dockerfile
This image requires to specify following build arguments:
| arg | description |
| --- | --- |
| `EMSCRIPTEN_VERSION` | One of released version of Emscripten. For example `1.38.45`<br/> Can be used with `-upstream` variant like: `1.38.45-upstream`<br /> Minimal supported version is **1.38.40**|
**Building**
This step will build Dockerfile as given tag on local machine
```bash
# using docker
docker build \
--build-arg=EMSCRIPTEN_VERSION=1.38.43-upstream \
--tag emscripten/emscripten:1.38.43-upstream \
.
```
```bash
# using predefined make target
make version=1.38.43-upstream build
```
**Tagging**
In case of using `docker build` command directly, given `--tag` should match version of released Emscripten (you can see list of non-legacy versions by executing `emsdk list`).
**Pushing**
This step will take local image and push to default docker registry. You need to make sure that you logged in docker cli (`docker login`) and you have rights to push to that registry.
```bash
# using docker
docker push emscripten/emscripten:1.38.43-upstream
```
```bash
# using predefined make target
make version=1.38.43-upstream push
```
In case of pushing the most recent version, this version should be also tagged as `latest` or `latest-upstream` and pushed.
```bash
# using docker cli
# in case of fastcomp variant (default backend)
docker tag emscripten/emscripten:1.38.43 emscripten/emscripten:latest
docker push emscripten/emscripten:latest
# in case of upstream variant
docker tag emscripten/emscripten:1.38.43-upstream emscripten/emscripten:latest-upstream
docker push emscripten/emscripten:latest-upstream
```
```bash
# using predefined make target
make version=1.38.43-upstream alias=latest-upstream push
```
### Extending
If your project uses packages that this image doesn't provide you might want to:
* Contribute to this repo: Maybe your dependency is either non-intrusive or could be useful for other people
* Create custom image that bases on this image
1. create own Dockerfile that holds:
```dockerfile
# Point at any base image that you find suitable to extend.
FROM emscripten/emscripten:1.38.25
# Install required tools that are useful for your project i.e. ninja-build
RUN apt update && apt install -y ninja-build
```
2. build it
```shell
docker build -t extended_emscripten .
```
3. test
```shell
docker run --rm extended_emscripten ninja --version
# Python 2.7.16
```

27
docker/entrypoint Executable file
View File

@@ -0,0 +1,27 @@
#!/bin/sh
# In case when mapped user id by `docker run -u` is not created inside docker image
# The `$HOME` variable points to `/` - which prevents any tool to write to, as it requires root access
# In such case we set `$HOME` to `/tmp` as it should r/w for everyone
if [ "$HOME" = "/" ] ; then
export HOME=/tmp
fi
# In case of running as root, use `umask` to reduce problem of file permission on host
if [ "$(id -g)" = "0" ] && [ "$(id -u)" = "0" ] ;
then
umask 0000
fi
# Export this image specific Environment variables
# Those variables are important to use dedicated folder for all cache and predefined config file
export EM_CONFIG=/emsdk/.emscripten
export EM_CACHE=/emsdk/.data/cache
export EM_PORTS=/emsdk/.data/ports
# Activate Emscripten SDK
. ${EMSDK}/emsdk_set_env.sh
# Evaluate a command that's coming after `docker run` / `docker exec`
"$@"

19
docker/test_dockerimage.sh Executable file
View File

@@ -0,0 +1,19 @@
#!/bin/bash
set -ex
which asm2wasm
which llvm-ar
which emsdk
node --version
npm --version
python3 --version
pip3 --version
python --version
pip --version
em++ --version
emcc --version
java -version
cmake --version
# cleanup after test
find ${EMSDK} -name "*.pyc" -exec rm {} \;