Build Linux binary in an older docker container (#268)
Currently our Linux binaries aren't quite as portable as they otherwise could be. There's two primary reasons for this, and one of them is that the binary is produced in a relatively recent Linux distribution (Ubuntu 16.04) which means it has a relatively recent requirement in terms of glibc versions. On OSX/Windows we can set some flags to rely on older libc implementations, but on Linux we have to actually build against an older version. This commit switches the container for the build to CentOS 6 instead of the default Ubuntu 16.04. The main trick here is also finding a C++11-capable compiler to compile wabt. Turns out though there's a helpful tutorial for this at https://edwards.sdsu.edu/research/c11-on-centos-6/ and it was as easy as installing a few packages. The second portability concern of our Linux binaries is that they link dynamically to `libstdc++.so` which isn't always installed on target systems, or even if it is it may be too old or have a different ABI. This is solved by statically linking to `libstdc++.a` in the build on Azure by doing a bit of trickery with libraries and what's available. After these results the glibc requirements drops from 2.18 (released in 2013) to 2.6 (released in 2007) and avoids the need for users to have libstdc++.so installed. We may eventually want to investigate fully-static musl binaries, but finding a musl compiler for C++ is something I'm not that good at, so I figure this is probably good enough for now.
This commit is contained in:
Alex Crichton
committed by
Till Schneidereit
Till Schneidereit
parent
3039caf65c
commit
47eee434d4
@@ -94,8 +94,6 @@ jobs:
|
|||||||
# Statically link against msvcrt to produce slightly more portable
|
# Statically link against msvcrt to produce slightly more portable
|
||||||
# binaries on Windows by reducing our binary compatibility requirements.
|
# binaries on Windows by reducing our binary compatibility requirements.
|
||||||
RUSTFLAGS: -Ctarget-feature=+crt-static
|
RUSTFLAGS: -Ctarget-feature=+crt-static
|
||||||
linux:
|
|
||||||
imageName: 'ubuntu-16.04'
|
|
||||||
mac:
|
mac:
|
||||||
imageName: 'macos-10.14'
|
imageName: 'macos-10.14'
|
||||||
# Lower the deployment target from our build image in an attempt to
|
# Lower the deployment target from our build image in an attempt to
|
||||||
@@ -103,121 +101,50 @@ jobs:
|
|||||||
# 10.9 here is arbitrarily chosen and just happens to be the lowest that
|
# 10.9 here is arbitrarily chosen and just happens to be the lowest that
|
||||||
# works at this time. Raising this is probably fine.
|
# works at this time. Raising this is probably fine.
|
||||||
MACOSX_DEPLOYMENT_TARGET: 10.9
|
MACOSX_DEPLOYMENT_TARGET: 10.9
|
||||||
|
|
||||||
variables:
|
variables:
|
||||||
toolchain: stable
|
toolchain: stable
|
||||||
|
|
||||||
pool:
|
pool:
|
||||||
vmImage: $(imageName)
|
vmImage: $(imageName)
|
||||||
|
|
||||||
# variables:
|
|
||||||
# SCCACHE_DIR: $(Pipeline.Workspace)/.sccache
|
|
||||||
# RUSTC_WRAPPER: sccache
|
|
||||||
|
|
||||||
steps:
|
steps:
|
||||||
- checkout: self
|
- template: ci/azure-build-release.yml
|
||||||
submodules: true
|
|
||||||
- template: ci/azure-install-rust.yml
|
|
||||||
|
|
||||||
- bash: echo "##vso[task.setvariable variable=RUSTC_VERSION;]`rustc --version`"
|
# Build the Linux release binary in an older Linux container (in this case
|
||||||
displayName: Set rustc version string for caching
|
# Centos 6)
|
||||||
|
- job: Build_linux
|
||||||
|
variables:
|
||||||
|
toolchain: stable
|
||||||
|
container:
|
||||||
|
image: centos:6
|
||||||
|
options: "--name ci-container -v /usr/bin/docker:/tmp/docker:ro"
|
||||||
|
steps:
|
||||||
|
# We're executing in the container as non-root but `yum` requires root. We
|
||||||
|
# need to install `sudo` but to do that we need `sudo`. Do a bit of a weird
|
||||||
|
# hack where we use the host `docker` executable to re-execute in our own
|
||||||
|
# container with the root user to install `sudo`
|
||||||
|
- bash: /tmp/docker exec -t -u 0 ci-container sh -c "yum install -y sudo"
|
||||||
|
displayName: Configure sudo
|
||||||
|
|
||||||
# - bash: |
|
# See https://edwards.sdsu.edu/research/c11-on-centos-6/ for where these
|
||||||
# set -e
|
# various commands came from.
|
||||||
# curl -Lfo sccache.tar.gz https://github.com/mozilla/sccache/releases/download/0.2.9/sccache-0.2.9-x86_64-apple-darwin.tar.gz
|
- bash: |
|
||||||
# tar -xzf sccache.tar.gz
|
set -e
|
||||||
# cp sccache-*/sccache /usr/local/bin/
|
sudo yum install -y centos-release-scl cmake xz
|
||||||
# displayName: Install sccache (OSX)
|
sudo yum install -y devtoolset-8-gcc devtoolset-8-binutils devtoolset-8-gcc-c++
|
||||||
# condition: and(succeeded(), eq(variables['Agent.OS'], 'Darwin'))
|
echo "##vso[task.prependpath]/opt/rh/devtoolset-8/root/usr/bin"
|
||||||
|
displayName: Install system dependencies
|
||||||
|
|
||||||
# - bash: |
|
# Delete `libstdc++.so` to force gcc to link against `libstdc++.a` instead.
|
||||||
# set -e
|
# This is a hack and not the right way to do this, but it ends up doing the
|
||||||
# curl -Lfo sccache.tar.gz https://github.com/mozilla/sccache/releases/download/0.2.9/sccache-0.2.9-x86_64-unknown-linux-musl.tar.gz
|
# right thing for now.
|
||||||
# tar -xzf sccache.tar.gz
|
- bash: sudo rm -f /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/libstdc++.so
|
||||||
# sudo cp sccache-*/sccache /usr/local/bin/
|
displayName: Force a static libstdc++
|
||||||
# displayName: Install sccache (Linux)
|
|
||||||
# condition: and(succeeded(), eq(variables['Agent.OS'], 'Linux'))
|
|
||||||
|
|
||||||
# - script: |
|
- template: ci/azure-build-release.yml
|
||||||
# curl -Lfo sccache.tar.gz https://github.com/mozilla/sccache/releases/download/0.2.9/sccache-0.2.9-x86_64-pc-windows-msvc.tar.gz
|
|
||||||
# tar -xzf sccache.tar.gz
|
|
||||||
# move sccache-* sccache
|
|
||||||
# echo "##vso[task.prependpath]%CD%\sccache"
|
|
||||||
# displayName: Install sccache (Windows)
|
|
||||||
# condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
|
|
||||||
|
|
||||||
- bash: cargo build --release
|
|
||||||
displayName: Cargo build
|
|
||||||
|
|
||||||
# Test what we're about to release in release mode itself. This tests
|
|
||||||
# everything except lightbeam which requires nightly which happens above.
|
|
||||||
- bash: cargo test --release --all --exclude lightbeam --exclude wasmtime-wasi-c
|
|
||||||
displayName: Cargo test
|
|
||||||
env:
|
|
||||||
RUST_BACKTRACE: 1
|
|
||||||
|
|
||||||
# - script: sccache --show-stats
|
|
||||||
# displayName: post-compile sccache stats
|
|
||||||
|
|
||||||
- bash: |
|
|
||||||
echo "##vso[task.setvariable variable=tagName;]`echo $BUILD_SOURCEBRANCH | sed -e 's|refs/tags/||'`"
|
|
||||||
displayName: Set tag name
|
|
||||||
condition: startsWith(variables['Build.SourceBranch'], 'refs/tags/')
|
|
||||||
- bash: |
|
|
||||||
echo "##vso[task.setvariable variable=tagName;]dev"
|
|
||||||
displayName: Set tag name to "dev"
|
|
||||||
condition: not(startsWith(variables['Build.SourceBranch'], 'refs/tags/'))
|
|
||||||
|
|
||||||
- bash: echo "##vso[task.setvariable variable=basename;]wasmtime-$(tagName)-x86_64-windows"
|
|
||||||
displayName: Configure basename var
|
|
||||||
condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
|
|
||||||
- bash: echo "##vso[task.setvariable variable=basename;]wasmtime-$(tagName)-x86_64-apple"
|
|
||||||
displayName: Configure basename var
|
|
||||||
condition: and(succeeded(), eq(variables['Agent.OS'], 'Darwin'))
|
|
||||||
- bash: echo "##vso[task.setvariable variable=basename;]wasmtime-$(tagName)-x86_64-linux"
|
|
||||||
displayName: Configure basename var
|
|
||||||
condition: and(succeeded(), eq( variables['Agent.OS'], 'Linux' ))
|
|
||||||
|
|
||||||
- bash: |
|
|
||||||
set -e
|
|
||||||
mkdir -p $BUILD_BINARIESDIRECTORY/$BASENAME
|
|
||||||
if [ "$AGENT_OS" = "Windows_NT" ]; then
|
|
||||||
ext=.exe
|
|
||||||
fi
|
|
||||||
cp LICENSE README.md target/release/{wasmtime,wasm2obj}$ext $BUILD_BINARIESDIRECTORY/$BASENAME
|
|
||||||
displayName: Copy binaries
|
|
||||||
|
|
||||||
- bash: |
|
|
||||||
set -e
|
|
||||||
export WT_VERSION=`cat Cargo.toml | sed -n 's/^version = "\([^"]*\)".*/\1/p'`
|
|
||||||
"$WIX/bin/candle" -arch x64 -out target/wasmtime.wixobj installer/msi/wasmtime.wxs
|
|
||||||
"$WIX/bin/light" -out $BUILD_ARTIFACTSTAGINGDIRECTORY/$(basename).msi target/wasmtime.wixobj -ext WixUtilExtension
|
|
||||||
rm $BUILD_ARTIFACTSTAGINGDIRECTORY/$(basename).wixpdb
|
|
||||||
displayName: Create installer (Windows)
|
|
||||||
condition: eq(variables['Agent.OS'], 'Windows_NT')
|
|
||||||
|
|
||||||
- task: ArchiveFiles@2
|
|
||||||
inputs:
|
|
||||||
rootFolderOrFile: $(Build.BinariesDirectory)/$(basename)
|
|
||||||
archiveType: 'zip'
|
|
||||||
archiveFile: '$(Build.ArtifactStagingDirectory)/$(basename).zip'
|
|
||||||
displayName: Archive files (Win)
|
|
||||||
condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
|
|
||||||
- task: ArchiveFiles@2
|
|
||||||
inputs:
|
|
||||||
rootFolderOrFile: $(Build.BinariesDirectory)/$(basename)
|
|
||||||
archiveType: 'tar'
|
|
||||||
tarCompression: 'xz'
|
|
||||||
archiveFile: '$(Build.ArtifactStagingDirectory)/$(basename).tar.xz'
|
|
||||||
displayName: Archive files (Unix)
|
|
||||||
condition: and(succeeded(), ne(variables['Agent.OS'], 'Windows_NT'))
|
|
||||||
- task: PublishPipelineArtifact@1
|
|
||||||
inputs:
|
|
||||||
path: $(Build.ArtifactStagingDirectory)/
|
|
||||||
artifactName: 'bundle-$(Agent.OS)'
|
|
||||||
|
|
||||||
- job: Publish
|
- job: Publish
|
||||||
dependsOn: Build
|
dependsOn:
|
||||||
|
- Build
|
||||||
|
- Build_linux
|
||||||
condition: and(succeeded(), in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI'))
|
condition: and(succeeded(), in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI'))
|
||||||
steps:
|
steps:
|
||||||
# Checking out the sources is needed to be able to delete the "dev" tag, see below.
|
# Checking out the sources is needed to be able to delete the "dev" tag, see below.
|
||||||
|
|||||||
103
ci/azure-build-release.yml
Normal file
103
ci/azure-build-release.yml
Normal file
@@ -0,0 +1,103 @@
|
|||||||
|
steps:
|
||||||
|
- checkout: self
|
||||||
|
submodules: true
|
||||||
|
|
||||||
|
- template: azure-install-rust.yml
|
||||||
|
|
||||||
|
- bash: echo "##vso[task.setvariable variable=RUSTC_VERSION;]`rustc --version`"
|
||||||
|
displayName: Set rustc version string for caching
|
||||||
|
|
||||||
|
# - bash: |
|
||||||
|
# set -e
|
||||||
|
# curl -Lfo sccache.tar.gz https://github.com/mozilla/sccache/releases/download/0.2.9/sccache-0.2.9-x86_64-apple-darwin.tar.gz
|
||||||
|
# tar -xzf sccache.tar.gz
|
||||||
|
# cp sccache-*/sccache /usr/local/bin/
|
||||||
|
# displayName: Install sccache (OSX)
|
||||||
|
# condition: and(succeeded(), eq(variables['Agent.OS'], 'Darwin'))
|
||||||
|
|
||||||
|
# - bash: |
|
||||||
|
# set -e
|
||||||
|
# curl -Lfo sccache.tar.gz https://github.com/mozilla/sccache/releases/download/0.2.9/sccache-0.2.9-x86_64-unknown-linux-musl.tar.gz
|
||||||
|
# tar -xzf sccache.tar.gz
|
||||||
|
# sudo cp sccache-*/sccache /usr/local/bin/
|
||||||
|
# displayName: Install sccache (Linux)
|
||||||
|
# condition: and(succeeded(), eq(variables['Agent.OS'], 'Linux'))
|
||||||
|
|
||||||
|
# - script: |
|
||||||
|
# curl -Lfo sccache.tar.gz https://github.com/mozilla/sccache/releases/download/0.2.9/sccache-0.2.9-x86_64-pc-windows-msvc.tar.gz
|
||||||
|
# tar -xzf sccache.tar.gz
|
||||||
|
# move sccache-* sccache
|
||||||
|
# echo "##vso[task.prependpath]%CD%\sccache"
|
||||||
|
# displayName: Install sccache (Windows)
|
||||||
|
# condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
|
||||||
|
|
||||||
|
- bash: cargo build --release
|
||||||
|
displayName: Cargo build
|
||||||
|
|
||||||
|
# Test what we're about to release in release mode itself. This tests
|
||||||
|
# everything except lightbeam which requires nightly which happens above.
|
||||||
|
- bash: cargo test --release --all --exclude lightbeam --exclude wasmtime-wasi-c
|
||||||
|
displayName: Cargo test
|
||||||
|
env:
|
||||||
|
RUST_BACKTRACE: 1
|
||||||
|
|
||||||
|
# - script: sccache --show-stats
|
||||||
|
# displayName: post-compile sccache stats
|
||||||
|
|
||||||
|
- bash: |
|
||||||
|
echo "##vso[task.setvariable variable=tagName;]`echo $BUILD_SOURCEBRANCH | sed -e 's|refs/tags/||'`"
|
||||||
|
displayName: Set tag name
|
||||||
|
condition: startsWith(variables['Build.SourceBranch'], 'refs/tags/')
|
||||||
|
- bash: |
|
||||||
|
echo "##vso[task.setvariable variable=tagName;]dev"
|
||||||
|
displayName: Set tag name to "dev"
|
||||||
|
condition: not(startsWith(variables['Build.SourceBranch'], 'refs/tags/'))
|
||||||
|
|
||||||
|
- bash: echo "##vso[task.setvariable variable=basename;]wasmtime-$(tagName)-x86_64-windows"
|
||||||
|
displayName: Configure basename var
|
||||||
|
condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
|
||||||
|
- bash: echo "##vso[task.setvariable variable=basename;]wasmtime-$(tagName)-x86_64-apple"
|
||||||
|
displayName: Configure basename var
|
||||||
|
condition: and(succeeded(), eq(variables['Agent.OS'], 'Darwin'))
|
||||||
|
- bash: echo "##vso[task.setvariable variable=basename;]wasmtime-$(tagName)-x86_64-linux"
|
||||||
|
displayName: Configure basename var
|
||||||
|
condition: and(succeeded(), eq( variables['Agent.OS'], 'Linux' ))
|
||||||
|
|
||||||
|
- bash: |
|
||||||
|
set -e
|
||||||
|
mkdir -p $BUILD_BINARIESDIRECTORY/$BASENAME
|
||||||
|
if [ "$AGENT_OS" = "Windows_NT" ]; then
|
||||||
|
ext=.exe
|
||||||
|
fi
|
||||||
|
cp LICENSE README.md target/release/{wasmtime,wasm2obj}$ext $BUILD_BINARIESDIRECTORY/$BASENAME
|
||||||
|
displayName: Copy binaries
|
||||||
|
|
||||||
|
- bash: |
|
||||||
|
set -e
|
||||||
|
export WT_VERSION=`cat Cargo.toml | sed -n 's/^version = "\([^"]*\)".*/\1/p'`
|
||||||
|
"$WIX/bin/candle" -arch x64 -out target/wasmtime.wixobj installer/msi/wasmtime.wxs
|
||||||
|
"$WIX/bin/light" -out $BUILD_ARTIFACTSTAGINGDIRECTORY/$(basename).msi target/wasmtime.wixobj -ext WixUtilExtension
|
||||||
|
rm $BUILD_ARTIFACTSTAGINGDIRECTORY/$(basename).wixpdb
|
||||||
|
displayName: Create installer (Windows)
|
||||||
|
condition: eq(variables['Agent.OS'], 'Windows_NT')
|
||||||
|
|
||||||
|
- task: ArchiveFiles@2
|
||||||
|
inputs:
|
||||||
|
rootFolderOrFile: $(Build.BinariesDirectory)/$(basename)
|
||||||
|
archiveType: 'zip'
|
||||||
|
archiveFile: '$(Build.ArtifactStagingDirectory)/$(basename).zip'
|
||||||
|
displayName: Archive files (Win)
|
||||||
|
condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
|
||||||
|
- task: ArchiveFiles@2
|
||||||
|
inputs:
|
||||||
|
rootFolderOrFile: $(Build.BinariesDirectory)/$(basename)
|
||||||
|
archiveType: 'tar'
|
||||||
|
tarCompression: 'xz'
|
||||||
|
archiveFile: '$(Build.ArtifactStagingDirectory)/$(basename).tar.xz'
|
||||||
|
displayName: Archive files (Unix)
|
||||||
|
condition: and(succeeded(), ne(variables['Agent.OS'], 'Windows_NT'))
|
||||||
|
- task: PublishPipelineArtifact@1
|
||||||
|
inputs:
|
||||||
|
path: $(Build.ArtifactStagingDirectory)/
|
||||||
|
artifactName: 'bundle-$(Agent.OS)'
|
||||||
|
|
||||||
Reference in New Issue
Block a user