This repository shows how to build Machine Learning pipeline for a vision model (TensorFlow) from ๐ค Transformers using the TensorFlow Ecosystem.
Chansung Park
This repository shows how to build Machine Learning pipeline for a vision model (TensorFlow) from ๐ค Transformers using the TensorFlow Ecosystem. In particular, we use TensorFlow Extended(TFX), and there are TensorFlow Data Validation(TFDV), Transform(TFT), Model Analysis(TFMA), and Serving(TF Serving) besides TensorFlow itself internally involved.
NOTE: This is a follow-up projects of "Deploying Vision Models (TensorFlow) from ๐ค Transformers" which shows how to deploy ViT model locally, on kubernetes, and on a fully managed service Vertex AI.
We will show how to build ML pipeline with TFX in a step-by-step manner:
ExampleGen
, Trainer
, and Pusher
. These components are responsible for injecting raw dataset into the ML pipeline, training a TensorFlow model, and deploying a trained model.
SchemaGen
, StatisticsGen
, and Transform
. These components are responsible for analyzing the structures of the dataset, analyzing the statistical traits of the features in the dataset, and data pre-processing.
Resolver
and Evaluator
. These components are responsible for importing existing Artifacts (such as previously trained model) and comparing the performance between two models (one from the Resolver
and one from the current pipeline run).
Tuner
. This component is responsible for running a set of experiments with different sets of hyperparameters with fewer epochs, and the found best hyperparameter combination will be passed to the Trainer
, and Trainer
will train the model longer time with that hyperparameter combinations as the starting point.
HFModelPusher
to push currently trained model to ๐ค Model Hub and HFSpacePusher
to automatically deploy Gradio application to ๐ค Space Hub.
We are thankful to the ML Developer Programs team at Google that provided GCP support.
We serve cookies on this site to analyze traffic, remember your preferences, and optimize your experience.