Python web scrapping on GCP

Content

build container images.

tag images.

upload images to GCP.

create services.

build container images.

selenium.

web scrapping.

tag images.

image with GCP pattern.

docker pull selenium/standalone-chrome
# tag with pattern $GCP_REGISTRY/$PROJECT_NAME/$IMAGE_NAME
docker tag selenium_standalone-chrome asia.gcr.io/$PROJECT_NAME/selenium-standalone-chrome
# push image
docker push asia.gcr.io/$PROJECT_NAME/selenium-standalone-chrome

upload images to GCP.

authentication on browser.

gcloud auth login

setup project.

gcloud config set project $PROJECT_NAME

create services.

Create from container images.

# create selenium service.
gcloud run deploy selenium-chrome --image asia.gcr.io/$PROJECT_NAME/selenium_standalone-chrome --port 4444 --memory 2G --region asia-southeast1 --platform managed

Create from cloud build.

# gcloud builds submit --tag asia.gcr.io/$PROJECT_NAME/$IMAGE_NAME

gcloud builds submit --tag asia.gcr.io/$PROJECT_NAME/web-scrpping
# deploy service
gcloud run deploy selenium-chrome --image asia.gcr.io/$PROJECT_NAME/selenium_standalone-chrome

Reference:
https://www.roelpeters.be/how-to-deploy-a-scraping-script-and-selenium-in-google-cloud-run/
https://dev.to/googlecloud/using-headless-chrome-with-cloud-run-3fdp