Skip to content

trigger build #86

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
e-davidson wants to merge 7 commits into
base: main
Choose a base branch
from
Open

trigger build #86

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ff2f839
trigger build
Jul 26, 2024
938c0e7
update release.json
Jul 26, 2024
0d77c66
bump rust version
Jul 26, 2024
6431656
update rust
Jul 26, 2024
054e1d5
update rust
Jul 26, 2024
5fd5a80
update rust-toolchain.toml for new rust version
Jul 28, 2024
dcf68c3
change test image
Jul 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions huggingface/pytorch/optimum/docker/0.0.22/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@ FROM alpine AS tgi
RUN mkdir -p /tgi
ADD https://github.com/huggingface/text-generation-inference/archive/refs/tags/v2.0.2.tar.gz /tgi/sources.tar.gz
RUN tar -C /tgi -xf /tgi/sources.tar.gz --strip-components=1
RUN sed -i 's/1.75.0/1.79.0/g' /tgi/rust-toolchain.toml

# Build cargo components (adapted from TGI original Dockerfile)
# Note that the build image is aligned on the same Linux version as the base image (Debian bookworm/ Ubuntu 22.04)
FROM lukemathwalker/cargo-chef:latest-rust-1.75-bookworm AS chef
FROM lukemathwalker/cargo-chef:latest-rust-1.79-bookworm AS chef
WORKDIR /usr/src

ARG CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse
Expand Down Expand Up @@ -37,6 +38,9 @@ COPY --from=tgi /tgi/proto proto
COPY --from=tgi /tgi/benchmark benchmark
COPY --from=tgi /tgi/router router
COPY --from=tgi /tgi/launcher launcher
RUN rustc --version
RUN rustup update
RUN rustc --version
RUN cargo build --release --workspace --exclude benchmark

# Fetch optimum-neuron sources
Expand Down Expand Up @@ -161,4 +165,4 @@ RUN echo "N.B.: Although this image is released under the Apache-2.0 License, th

LABEL dlc_major_version="1"
LABEL com.amazonaws.ml.engines.sagemaker.dlc.framework.huggingface.tgi="true"
LABEL com.amazonaws.sagemaker.capabilities.accept-bind-to-port="true"
LABEL com.amazonaws.sagemaker.capabilities.accept-bind-to-port="true"
9 changes: 4 additions & 5 deletions releases.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
{
"device": "inf2",
"min_version": "0.0.22",
"max_version": "0.0.23",
"max_version": "0.0.22",
"os_version": "ubuntu22.04",
"python_version": "py310",
"pytorch_version": "2.1.2"
Expand Down Expand Up @@ -79,12 +79,11 @@
"releases": [
{
"framework": "TGI",
"device": "gpu",
"version": "2.2.0",
"device": "inf2",
"version": "0.0.22",
"os_version": "ubuntu22.04",
"cuda_version": "cu121",
"python_version": "py310",
"pytorch_version": "2.3.0"
"pytorch_version": "2.1.2"
}
]
}
2 changes: 1 addition & 1 deletion tests/huggingface/sagemaker_dlc_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ def get_models_for_image(image_type, device_type):
("google/flan-t5-xxl", None, "ml.g5.12xlarge"),
]
elif device_type == "inf2":
return [ ("princeton-nlp/Sheared-LLaMA-1.3B", None, "ml.inf2.xlarge") ]
return [ ("HuggingFaceTB/cosmo-1b", None, "ml.inf2.8xlarge") ]
else:
raise ValueError(f"No testing models found for {image_type} on instance {device_type}. "
f"please check whether the image_type and instance_type are supported.")
Expand Down