Skip to content

Upgrade rustc deps, use environment cargo path #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Upgrade rustc deps, use environment cargo path #8

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions setup_scripts/conda_server.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
DIR="$(dirname "$DIR")"

# define colors for output
COLOR_GREEN='\033[0;32m'
COLOR_NC='\033[0m'
COLOR_RED='\033[0;31m'

# currently can only build in TIR without extensions
# seems un-important, as it only affects BLOOM/NEOX
ENV_NAME=tgi-env
Expand Down Expand Up @@ -90,7 +95,7 @@ conda activate ${ENV_NAME}
# # Install dependencies
mamba install -y -c conda-forge coreutils "gxx<12.0"
mamba install -y -c conda-forge curl git tar
mamba install -y -c conda-forge "rust>=1.65.0"
mamba install -y -c conda-forge "rust>=1.74"
mamba install -y -c conda-forge openssh
mamba install -y -c "nvidia/label/cuda-11.8.0" cuda-toolkit
# pin pytorch due to some cuda-issue in pytorch==2.1.0 / something with vllm
Expand All @@ -102,9 +107,6 @@ export LD_LIBRARY_PATH=${CONDA_HOME}/envs/${ENV_NAME}/lib:$LD_LIBRARY_PATH
export PATH=${CONDA_HOME}/envs/${ENV_NAME}/bin:$PATH
export CUDA_HOME=${CONDA_HOME}/envs/${ENV_NAME}

# add cargo bin
export PATH=~/.cargo/bin:$PATH

# add protoc
export PROTOC_ZIP=protoc-21.12-linux-x86_64.zip
mkdir -p /tmp/protoc
Expand Down Expand Up @@ -154,6 +156,8 @@ if [ "$BUILD_VLLM" = true ] ; then
rm -rf workdir/*
fi

echo -e "${COLOR_GREEN}DONE INSTALLING VLLM${COLOR_NC}"

# install base package
cd ${DIR}
OPENSSL_DIR=${DIR}/.openssl \
Expand All @@ -162,6 +166,7 @@ OPENSSL_INCLUDE_DIR=${DIR}/.openssl/include \
BUILD_EXTENSIONS=$BUILD_EXTENSIONS \
make install

echo -e "${COLOR_GREEN}DONE INSTALLING BASE PACKAGE${COLOR_NC}"

# install flash attention
if [ "$BUILD_FLASHATTN" = true ] ; then
Expand All @@ -177,6 +182,8 @@ fi

rm -rf workdir

echo -e "${COLOR_GREEN}DONE INSTALLING FLASH ATTENTION${COLOR_NC}"

# override protobuf
pip install 'protobuf<3.21'

Expand Down